* [PATCH v4 00/15] Enable Memory Bandwidth Allocation in Xen
@ 2017-09-23 9:48 Yi Sun
2017-09-23 9:48 ` [PATCH v4 01/15] docs: create Memory Bandwidth Allocation (MBA) feature document Yi Sun
` (16 more replies)
0 siblings, 17 replies; 64+ messages in thread
From: Yi Sun @ 2017-09-23 9:48 UTC (permalink / raw)
To: xen-devel
Cc: Yi Sun, Andrew Cooper, Ian Jackson, Julien Grall, Jan Beulich,
Chao Peng, Wei Liu, Daniel De Graaf, Roger Pau Monné
Hi, all,
We plan to bring a new PSR (Platform Shared Resource) feature called
Intel Memory Bandwidth Allocation (MBA) to Xen.
Besides the MBA enabling, we change some interfaces to make them more
general but not only for CAT.
Any comments are welcome!
You can find this series at:
https://github.com/yisun-git/xen_mba mba_v4
CC: Jan Beulich <jbeulich@suse.com>
CC: Andrew Cooper <andrew.cooper3@citrix.com>
CC: Wei Liu <wei.liu2@citrix.com>
CC: Ian Jackson <ian.jackson@eu.citrix.com>
CC: Daniel De Graaf <dgdegra@tycho.nsa.gov>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
CC: Chao Peng <chao.p.peng@linux.intel.com>
CC: Julien Grall <julien.grall@arm.com>
---
Acked and Reviewed list before V4:
a - Acked-by
r - Reviewed-by
ar patch 2 - Rename PSR sysctl/domctl interfaces and xsm policy to make them be general
r patch 3 - x86: rename 'cbm_type' to 'psr_type' to make it general
r patch 6 - x86: implement get value interface for MBA
ar patch 12 - tools: rename 'xc_psr_cat_type' to 'xc_psr_type'
a patch 13 - tools: implement new generic get value interface and MBA get value command
a patch 15 - docs: add MBA description in docs
---
V4 change list:
Patch 1:
- add 'domain-name' as parameter of 'psr-mba-show/psr-mba-set'.
(suggested by Roger Pau Monné)
- fix some wordings.
(suggested by Roger Pau Monné)
- explain how user can know the MBA_MAX.
(suggested by Roger Pau Monné)
- move the description of 'Linear mode/Non-linear mode' into section
of 'psr-mba-show'.
(suggested by Roger Pau Monné)
- change 'per-thread' to 'per-hyper-thread' to make it clearer.
(suggested by Roger Pau Monné)
- upgrade revision number.
Patch 2:
- remove 'ALLOC_' from names.
(suggested by Roger Pau Monné)
- fix comments.
(suggested by Roger Pau Monné)
Patch 3:
- fix words in commit message.
(suggested by Roger Pau Monné)
Patch 4:
- modify commit message.
(suggested by Roger Pau Monné)
- fix a comment.
(suggested by Roger Pau Monné)
- join two checks in a single if.
(suggested by Roger Pau Monné)
- remove redundant initialization of 'feat->cos_reg_val[0]'.
(suggested by Roger Pau Monné)
- change 'reg_b' to 'ebx'.
(suggested by Jan Beulich)
- change type of 'mba_init_feature' from 'int' to 'bool'.
(suggested by Roger Pau Monné)
- change type of 'cat_init_feature' from 'int' to 'bool'.
Patch 5:
- remove 'ALLOC_' from macro names.
(suggested by Roger Pau Monné)
- initialize 'data[PSR_INFO_ARRAY_SIZE]' to 0 to prevent to leak stack data.
(suggested by Roger Pau Monné)
Patch 6:
- remove 'ALLOC_' from macro names.
(suggested by Roger Pau Monné)
Patch 7:
- remove 'ALLOC_' from macro names.
(suggested by Roger Pau Monné)
- join two checks into a single if.
(suggested by Roger Pau Monné)
- remove redundant local variable 'array_len'.
(suggested by Roger Pau Monné)
Patch 8:
- add description for LIBXL_HAVE_PSR_GENERIC to mention newly added
public functions.
(suggested by Roger Pau Monné)
Patch 9:
- remove 'ALLOC_' from macro names.
(suggested by Roger Pau Monné)
- remove 'XC_PSR_FEAT_UNKNOWN' which is not necessary.
(suggested by Roger Pau Monné)
- remove 'FEAT_' from enum item names.
(suggested by Roger Pau Monné)
- remove 'xc_' from struct name.
(suggested by Roger Pau Monné)
- adjust codes to reduce indentation.
(suggested by Roger Pau Monné)
- assert for not happened case.
(suggested by Roger Pau Monné)
- add LOGE to show errno.
(suggested by Roger Pau Monné)
Patch 10:
- remove 'xc_' from struct name.
(suggested by Roger Pau Monné)
- fix words in commit message.
(suggested by Roger Pau Monné)
- change type of 'libxl__hw_info_to_libxl_cat_info' to void and use
assert to check invalid type. Then, remove check for
'libxl__hw_info_to_libxl_cat_info'.
(suggested by Roger Pau Monné)
- change type of 'libxl__xc_hw_info_to_libxl_hw_info' to void and use
assert to check invalid type. Then, remove check for
'libxl__xc_hw_info_to_libxl_hw_info'.
(suggested by Roger Pau Monné)
Patch 12:
- move assignment of xc_type to its declaration place.
(suggested by Roger Pau Monné)
Patch 13:
- use designated initializers for 'feat_name[]'.
(suggested by Roger Pau Monné)
- use LOG in 'libxl__psr_alloc_log_err_msg'.
(suggested by Roger Pau Monné)
Patch 14:
- remove 'ALLOC_' from macro name.
(suggested by Roger Pau Monné)
- adjust place of argc check and return EXIT_FAILURE.
(suggested by Roger Pau Monné)
- fix indentation issue.
(suggested by Roger Pau Monné)
- move same type local variables declaration to a single line.
(suggested by Roger Pau Monné)
Patch 15:
- modify description of MBA in 'xl.pod.1.in' to be same as feature doc.
(suggested by Roger Pau Monné)
- fix words issue.
(suggested by Roger Pau Monné)
Yi Sun (15):
docs: create Memory Bandwidth Allocation (MBA) feature document
Rename PSR sysctl/domctl interfaces and xsm policy to make them be
general
x86: rename 'cbm_type' to 'psr_type' to make it general
x86: implement data structure and CPU init flow for MBA
x86: implement get hw info flow for MBA
x86: implement get value interface for MBA
x86: implement set value flow for MBA
tools: create general interfaces to support psr allocation features
tools: implement the new libxc get hw info interface
tools: implement the new libxl get hw info interface
tools: implement the new xl get hw info interface
tools: rename 'xc_psr_cat_type' to 'xc_psr_type'
tools: implement new generic get value interface and MBA get value
command
tools: implement new generic set value interface and MBA set value
command
docs: add MBA description in docs
docs/features/intel_psr_mba.pandoc | 291 +++++++++++++++++++++++++++++
docs/man/xl.pod.1.in | 33 ++++
docs/misc/xl-psr.markdown | 63 +++++++
tools/flask/policy/modules/dom0.te | 4 +-
tools/libxc/include/xenctrl.h | 46 +++--
tools/libxc/xc_psr.c | 114 +++++++-----
tools/libxl/libxl.h | 37 ++++
tools/libxl/libxl_psr.c | 219 +++++++++++++++++-----
tools/libxl/libxl_types.idl | 22 +++
tools/xl/xl.h | 2 +
tools/xl/xl_cmdtable.c | 12 ++
tools/xl/xl_psr.c | 280 ++++++++++++++++++++++------
xen/arch/x86/domctl.c | 81 ++++----
xen/arch/x86/psr.c | 358 +++++++++++++++++++++++++-----------
xen/arch/x86/sysctl.c | 53 ++++--
xen/include/asm-x86/msr-index.h | 1 +
xen/include/asm-x86/psr.h | 22 ++-
xen/include/public/domctl.h | 32 ++--
xen/include/public/sysctl.h | 28 ++-
xen/xsm/flask/hooks.c | 8 +-
xen/xsm/flask/policy/access_vectors | 8 +-
21 files changed, 1354 insertions(+), 360 deletions(-)
create mode 100644 docs/features/intel_psr_mba.pandoc
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* [PATCH v4 01/15] docs: create Memory Bandwidth Allocation (MBA) feature document
2017-09-23 9:48 [PATCH v4 00/15] Enable Memory Bandwidth Allocation in Xen Yi Sun
@ 2017-09-23 9:48 ` Yi Sun
2017-09-25 15:13 ` Roger Pau Monné
2017-09-23 9:48 ` [PATCH v4 02/15] Rename PSR sysctl/domctl interfaces and xsm policy to make them be general Yi Sun
` (15 subsequent siblings)
16 siblings, 1 reply; 64+ messages in thread
From: Yi Sun @ 2017-09-23 9:48 UTC (permalink / raw)
To: xen-devel
Cc: Yi Sun, Andrew Cooper, Ian Jackson, Julien Grall, Jan Beulich,
Chao Peng, Wei Liu, Daniel De Graaf, Roger Pau Monné
This patch creates MBA feature document in doc/features/. It describes
key points to implement MBA which is described in details in Intel SDM
"Introduction to Memory Bandwidth Allocation".
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
---
CC: Jan Beulich <jbeulich@suse.com>
CC: Andrew Cooper <andrew.cooper3@citrix.com>
CC: Wei Liu <wei.liu2@citrix.com>
CC: Ian Jackson <ian.jackson@eu.citrix.com>
CC: Daniel De Graaf <dgdegra@tycho.nsa.gov>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
CC: Chao Peng <chao.p.peng@linux.intel.com>
CC: Julien Grall <julien.grall@arm.com>
v4:
- add 'domain-name' as parameter of 'psr-mba-show/psr-mba-set'.
(suggested by Roger Pau Monné)
- fix some wordings.
(suggested by Roger Pau Monné)
- explain how user can know the MBA_MAX.
(suggested by Roger Pau Monné)
- move the description of 'Linear mode/Non-linear mode' into section
of 'psr-mba-show'.
(suggested by Roger Pau Monné)
- change 'per-thread' to 'per-hyper-thread' to make it clearer.
(suggested by Roger Pau Monné)
- upgrade revision number.
v3:
- remove 'closed-loop' related description.
(suggested by Roger Pau Monné)
- explain 'linear' and 'non-linear' before mentioning them.
(suggested by Roger Pau Monné)
- adjust desription of 'psr-mba-set'.
(suggested by Roger Pau Monné)
- explain 'MBA_MAX'.
(suggested by Roger Pau Monné)
- remove 'n<64'.
(suggested by Roger Pau Monné)
- fix some wordings.
(suggested by Roger Pau Monné)
- add context in 'Testing' part to make things more clear.
(suggested by Roger Pau Monné)
v2:
- declare 'HW' in Terminology.
(suggested by Chao Peng)
- replace 'COS ID of VCPU' to 'COS ID of domain'.
(suggested by Chao Peng)
- replace 'COS register' to 'Thrtl MSR'.
(suggested by Chao Peng)
- add description for 'psr-mba-show' to state that the decimal value is
shown for linear mode but hexadecimal value is shown for non-linear mode.
(suggested by Chao Peng)
- remove content in 'Areas for improvement'.
(suggested by Chao Peng)
- use '<>' to specify mandatory argument to a command.
(suggested by Wei Liu)
v1:
- remove a special character to avoid the error when building pandoc.
---
docs/features/intel_psr_mba.pandoc | 291 +++++++++++++++++++++++++++++++++++++
1 file changed, 291 insertions(+)
create mode 100644 docs/features/intel_psr_mba.pandoc
diff --git a/docs/features/intel_psr_mba.pandoc b/docs/features/intel_psr_mba.pandoc
new file mode 100644
index 0000000..7a6a588
--- /dev/null
+++ b/docs/features/intel_psr_mba.pandoc
@@ -0,0 +1,291 @@
+% Intel Memory Bandwidth Allocation (MBA) Feature
+% Revision 1.6
+
+\clearpage
+
+# Basics
+
+---------------- ----------------------------------------------------
+ Status: **Tech Preview**
+
+Architecture(s): Intel x86
+
+ Component(s): Hypervisor, toolstack
+
+ Hardware: MBA is supported on Skylake Server and beyond
+---------------- ----------------------------------------------------
+
+# Terminology
+
+* CAT Cache Allocation Technology
+* CBM Capacity BitMasks
+* CDP Code and Data Prioritization
+* COS/CLOS Class of Service
+* HW Hardware
+* MBA Memory Bandwidth Allocation
+* MSRs Machine Specific Registers
+* PSR Intel Platform Shared Resource
+* THRTL Throttle value or delay value
+
+# Overview
+
+The Memory Bandwidth Allocation (MBA) feature provides indirect and approximate
+control over memory bandwidth available per-core. This feature provides OS/
+hypervisor the ability to slow misbehaving apps/domains by using a credit-based
+throttling mechanism.
+
+# User details
+
+* Feature Enabling:
+
+ Add "psr=mba" to boot line parameter to enable MBA feature.
+
+* xl interfaces:
+
+ 1. `psr-mba-show [domain-id|domain-name]`:
+
+ Show memory bandwidth throttling for domain. Under different modes, it
+ shows different type of data.
+
+ There are two modes:
+ Linear mode: the input precision is defined as 100-(MBA_MAX). For instance,
+ if the MBA_MAX value is 90, the input precision is 10%. Values not an even
+ multiple of the precision (e.g., 12%) will be rounded down (e.g., to 10%
+ delay applied) by HW automatically. The response of throttling value is
+ linear.
+
+ Non-linear mode: input delay values are powers-of-two from zero to the
+ MBA_MAX value from CPUID. In this case any values not a power of two will
+ be rounded down the next nearest power of two by HW automatically. The
+ response of throttling value is non-linear.
+
+ For linear mode, it shows the decimal value. For non-linear mode, it shows
+ hexadecimal value.
+
+ 2. `psr-mba-set [OPTIONS] <domain-id|domain-name> <throttling>`:
+
+ Set memory bandwidth throttling for domain.
+
+ Options:
+ '-s': Specify the socket to process, otherwise all sockets are processed.
+
+ Throttling value set in register implies the approximate amount of delaying
+ the traffic between core and memory. The higher throttling value results in
+ lower bandwidth. The max throttling value (MBA_MAX) supported can be
+ obtained through CPUID inside hypervisor. User can know it through
+ `psr-hwinfo`.
+
+# Technical details
+
+MBA is a member of Intel PSR features, it shares the base PSR infrastructure
+in Xen.
+
+## Hardware perspective
+
+ MBA defines a range of MSRs to support specifying a delay value (Thrtl) per
+ COS, with details below.
+
+ ```
+ +----------------------------+----------------+
+ | MSR (per socket) | Address |
+ +----------------------------+----------------+
+ | IA32_L2_QOS_Ext_BW_Thrtl_0 | 0xD50 |
+ +----------------------------+----------------+
+ | ... | ... |
+ +----------------------------+----------------+
+ | IA32_L2_QOS_Ext_BW_Thrtl_n | 0xD50+n |
+ +----------------------------+----------------+
+ ```
+
+ When context switch happens, the COS ID of domain is written to per-hyper-
+ thread MSR `IA32_PQR_ASSOC`, and then hardware enforces bandwidth allocation
+ according to the throttling value stored in the Thrtl MSR register.
+
+## The relationship between MBA and CAT/CDP
+
+ Generally speaking, MBA is completely independent of CAT/CDP, and any
+ combination may be applied at any time, e.g. enabling MBA with CAT
+ disabled.
+
+ But it needs to be noticed that MBA shares COS infrastructure with CAT,
+ although MBA is enumerated by different CPUID leaf from CAT (which
+ indicates that the max COS of MBA may be different from CAT). In some
+ cases, a domain is permitted to have a COS that is beyond one (or more)
+ of PSR features but within the others. For instance, let's assume the max
+ COS of MBA is 8 but the max COS of L3 CAT is 16, when a domain is assigned
+ 9 as COS, the L3 CAT CBM associated to COS 9 would be enforced, but for MBA,
+ the HW works as default value is set since COS 9 is beyond the max COS (8)
+ of MBA.
+
+## Design Overview
+
+* Core COS/Thrtl association
+
+ When enforcing Memory Bandwidth Allocation, all cores of domains have
+ the same default Thrtl MSR (COS0) which stores the same Thrtl (0). The
+ default Thrtl MSR is used only in hypervisor and is transparent to tool stack
+ and user.
+
+ System administrators can change PSR allocation policy at runtime by
+ using the tool stack. Since MBA shares COS ID with CAT/CDP, a COS ID
+ corresponds to a 2-tuple, like [CBM, Thrtl] with only-CAT enabled, when CDP
+ is enabled, the COS ID corresponds to a 3-tuple, like [Code_CBM, Data_CBM,
+ Thrtl]. If neither CAT nor CDP is enabled, things are easier, since one COS
+ ID corresponds to one Thrtl.
+
+* VCPU schedule
+
+ This part reuses CAT COS infrastructure.
+
+* Multi-sockets
+
+ Different sockets may have different MBA ability (like max COS)
+ although it is consistent on the same socket. So the capability
+ of per-socket MBA is specified.
+
+ This part reuses CAT COS infrastructure.
+
+## Implementation Description
+
+* Hypervisor interfaces:
+
+ 1. Boot line param: "psr=mba" to enable the feature.
+
+ 2. SYSCTL:
+ - XEN_SYSCTL_PSR_MBA_get_info: Get system MBA information.
+
+ 3. DOMCTL:
+ - XEN_DOMCTL_PSR_MBA_OP_GET_THRTL: Get throttling for a domain.
+ - XEN_DOMCTL_PSR_MBA_OP_SET_THRTL: Set throttling for a domain.
+
+* xl interfaces:
+
+ 1. psr-mba-show [domain-id]
+ Show system/domain runtime MBA throttling value. For linear mode,
+ it shows the decimal value. For non-linear mode, it shows hexadecimal
+ value.
+ => XEN_SYSCTL_PSR_MBA_get_info/XEN_DOMCTL_PSR_MBA_OP_GET_THRTL
+
+ 2. psr-mba-set [OPTIONS] <domain-id> <throttling>
+ Set bandwidth throttling for a domain.
+ => XEN_DOMCTL_PSR_MBA_OP_SET_THRTL
+
+ 3. psr-hwinfo
+ Show PSR HW information, including L3 CAT/CDP/L2 CAT/MBA.
+ => XEN_SYSCTL_PSR_MBA_get_info
+
+* Key data structure:
+
+ 1. Feature HW info
+
+ ```
+ struct {
+ unsigned int thrtl_max;
+ bool linear;
+ } mba;
+
+ - Member `thrtl_max`
+
+ `thrtl_max` is the max throttling value to be set, i.e. MBA_MAX.
+
+ - Member `linear`
+
+ `linear` means the response of delay value is linear or not.
+
+ As mentioned above, MBA is a member of Intel PSR features, it would
+ share the base PSR infrastructure in Xen. For example, the 'cos_max'
+ is a common HW property for all features. So, for other data structure
+ details, please refer to 'intel_psr_cat_cdp.pandoc'.
+
+# Limitations
+
+MBA can only work on HW which supports it (check by CPUID).
+
+# Testing
+
+We can execute these commands to verify MBA on different HWs supporting them.
+
+For example:
+ 1. User can get the MBA hardware info through 'psr-hwinfo' command. From
+ result, user can know if this hardware works under linear mode or non-
+ linear mode, the max throttling value (MBA_MAX) and so on.
+
+ root@:~$ xl psr-hwinfo --mba
+ Memory Bandwidth Allocation (MBA):
+ Socket ID : 0
+ Linear Mode : Enabled
+ Maximum COS : 7
+ Maximum Throttling Value: 90
+ Default Throttling Value: 0
+
+ 2. Then, user can set a throttling value to a domain. For example, set '0xa',
+ i.e 10% delay.
+
+ root@:~$ xl psr-mba-set 1 0xa
+
+ 3. User can check the current configuration of the domain through
+ 'psr-mab-show'. For linear mode, the decimal value is shown.
+
+ root@:~$ xl psr-mba-show 1
+ Socket ID : 0
+ Default THRTL : 0
+ ID NAME THRTL
+ 1 ubuntu14 10
+
+# Areas for improvement
+
+N/A
+
+# Known issues
+
+N/A
+
+# References
+
+"INTEL RESOURCE DIRECTOR TECHNOLOGY (INTEL RDT) ALLOCATION FEATURES" [Intel 64 and IA-32 Architectures Software Developer Manuals, vol3](http://www.intel.com/content/www/us/en/processors/architectures-software-developer-manuals.html)
+
+# History
+
+------------------------------------------------------------------------
+Date Revision Version Notes
+---------- -------- -------- -------------------------------------------
+2017-01-10 1.0 Xen 4.9 Design document written
+2017-07-10 1.1 Xen 4.10 Changes:
+ 1. Modify data structure according to latest
+ codes;
+ 2. Add content for 'Areas for improvement';
+ 3. Other minor changes.
+2017-08-09 1.2 Xen 4.10 Changes:
+ 1. Remove a special character to avoid error when
+ building pandoc.
+2017-08-15 1.3 Xen 4.10 Changes:
+ 1. Add terminology 'HW'.
+ 2. Change 'COS ID of VCPU' to 'COS ID of domain'.
+ 3. Change 'COS register' to 'Thrtl MSR'.
+ 4. Explain the value shown for 'psr-mba-show' under
+ different modes.
+ 5. Remove content in 'Areas for improvement'.
+2017-08-16 1.4 Xen 4.10 Changes:
+ 1. Add '<>' for mandatory argument.
+2017-08-30 1.5 Xen 4.10 Changes:
+ 1. Modify words in 'Overview' to make it easier to
+ understand.
+ 2. Explain 'linear/non-linear' modes before mention
+ them.
+ 3. Explain throttling value more accurate.
+ 4. Explain 'MBA_MAX'.
+ 5. Correct some words in 'Design Overview'.
+ 6. Change 'mba_info' to 'mba' according to code
+ changes. Also, modify contents of it.
+ 7. Add context in 'Testing' part to make things
+ more clear.
+ 8. Remove 'n<64' to avoid out-of-sync.
+2017-09-21 1.6 Xen 4.10 Changes:
+ 1. Add 'domain-name' as parameter of 'psr-mba-show/
+ psr-mba-set'.
+ 2. Fix some wordings.
+ 3. Explain how user can know the MBA_MAX.
+ 4. Move the description of 'Linear mode/Non-linear
+ mode' into section of 'psr-mba-show'.
+ 5. Change 'per-thread' to 'per-hyper-thread'.
+---------- -------- -------- -------------------------------------------
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v4 02/15] Rename PSR sysctl/domctl interfaces and xsm policy to make them be general
2017-09-23 9:48 [PATCH v4 00/15] Enable Memory Bandwidth Allocation in Xen Yi Sun
2017-09-23 9:48 ` [PATCH v4 01/15] docs: create Memory Bandwidth Allocation (MBA) feature document Yi Sun
@ 2017-09-23 9:48 ` Yi Sun
2017-09-25 16:15 ` Roger Pau Monné
` (2 more replies)
2017-09-23 9:48 ` [PATCH v4 03/15] x86: rename 'cbm_type' to 'psr_type' to make it general Yi Sun
` (14 subsequent siblings)
16 siblings, 3 replies; 64+ messages in thread
From: Yi Sun @ 2017-09-23 9:48 UTC (permalink / raw)
To: xen-devel
Cc: Yi Sun, Andrew Cooper, Ian Jackson, Jan Beulich, Chao Peng,
Wei Liu, Daniel De Graaf, Roger Pau Monné
This patch renames PSR sysctl/domctl interfaces and related xsm policy to
make them be general for all resource allocation features but not only
for CAT. Then, we can resuse the interfaces for all allocation features.
Basically, it changes 'psr_cat_op' to 'psr_alloc', and remove 'CAT_' from some
macros. E.g.:
1. psr_cat_op -> psr_alloc
2. XEN_DOMCTL_psr_cat_op -> XEN_DOMCTL_psr_alloc
3. XEN_SYSCTL_psr_cat_op -> XEN_SYSCTL_psr_alloc
4. XEN_DOMCTL_PSR_CAT_SET_L3_CBM -> XEN_DOMCTL_PSR_SET_L3_CBM
5. XEN_SYSCTL_PSR_CAT_get_l3_info -> XEN_SYSCTL_PSR_get_l3_info
The sysctl/domctl version numbers are bumped.
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Daniel De Graaf <dgdegra@tycho.nsa.gov>
---
CC: Jan Beulich <jbeulich@suse.com>
CC: Andrew Cooper <andrew.cooper3@citrix.com>
CC: Wei Liu <wei.liu2@citrix.com>
CC: Ian Jackson <ian.jackson@eu.citrix.com>
CC: Daniel De Graaf <dgdegra@tycho.nsa.gov>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Chao Peng <chao.p.peng@linux.intel.com>
v4:
- remove 'ALLOC_' from names.
(suggested by Roger Pau Monné)
- fix comments.
(suggested by Roger Pau Monné)
v3:
- remove 'op/OP' from names and modify some names from 'PSR_CAT' to
'PSR_ALLOC'.
(suggested by Roger Pau Monné)
v1:
- add description about what to be changed in commit message.
(suggested by Wei Liu)
- bump sysctl/domctl version numbers.
(suggested by Wei Liu)
---
tools/flask/policy/modules/dom0.te | 4 +--
tools/libxc/xc_psr.c | 52 ++++++++++++++++++-------------------
xen/arch/x86/domctl.c | 52 ++++++++++++++++++-------------------
xen/arch/x86/psr.c | 2 +-
xen/arch/x86/sysctl.c | 28 ++++++++++----------
xen/include/public/domctl.h | 30 ++++++++++-----------
xen/include/public/sysctl.h | 20 +++++++-------
xen/xsm/flask/hooks.c | 8 +++---
xen/xsm/flask/policy/access_vectors | 8 +++---
9 files changed, 102 insertions(+), 102 deletions(-)
diff --git a/tools/flask/policy/modules/dom0.te b/tools/flask/policy/modules/dom0.te
index 338caaf..fb1a299 100644
--- a/tools/flask/policy/modules/dom0.te
+++ b/tools/flask/policy/modules/dom0.te
@@ -14,7 +14,7 @@ allow dom0_t xen_t:xen {
tmem_control getscheduler setscheduler
};
allow dom0_t xen_t:xen2 {
- resource_op psr_cmt_op psr_cat_op pmu_ctrl get_symbol
+ resource_op psr_cmt_op psr_alloc pmu_ctrl get_symbol
get_cpu_levelling_caps get_cpu_featureset livepatch_op
gcov_op set_parameter
};
@@ -39,7 +39,7 @@ allow dom0_t dom0_t:domain {
};
allow dom0_t dom0_t:domain2 {
set_cpuid gettsc settsc setscheduler set_max_evtchn set_vnumainfo
- get_vnumainfo psr_cmt_op psr_cat_op
+ get_vnumainfo psr_cmt_op psr_alloc
};
allow dom0_t dom0_t:resource { add remove };
diff --git a/tools/libxc/xc_psr.c b/tools/libxc/xc_psr.c
index 039b920..623e26b 100644
--- a/tools/libxc/xc_psr.c
+++ b/tools/libxc/xc_psr.c
@@ -258,27 +258,27 @@ int xc_psr_cat_set_domain_data(xc_interface *xch, uint32_t domid,
switch ( type )
{
case XC_PSR_CAT_L3_CBM:
- cmd = XEN_DOMCTL_PSR_CAT_OP_SET_L3_CBM;
+ cmd = XEN_DOMCTL_PSR_SET_L3_CBM;
break;
case XC_PSR_CAT_L3_CBM_CODE:
- cmd = XEN_DOMCTL_PSR_CAT_OP_SET_L3_CODE;
+ cmd = XEN_DOMCTL_PSR_SET_L3_CODE;
break;
case XC_PSR_CAT_L3_CBM_DATA:
- cmd = XEN_DOMCTL_PSR_CAT_OP_SET_L3_DATA;
+ cmd = XEN_DOMCTL_PSR_SET_L3_DATA;
break;
case XC_PSR_CAT_L2_CBM:
- cmd = XEN_DOMCTL_PSR_CAT_OP_SET_L2_CBM;
+ cmd = XEN_DOMCTL_PSR_SET_L2_CBM;
break;
default:
errno = EINVAL;
return -1;
}
- domctl.cmd = XEN_DOMCTL_psr_cat_op;
+ domctl.cmd = XEN_DOMCTL_psr_alloc;
domctl.domain = (domid_t)domid;
- domctl.u.psr_cat_op.cmd = cmd;
- domctl.u.psr_cat_op.target = target;
- domctl.u.psr_cat_op.data = data;
+ domctl.u.psr_alloc.cmd = cmd;
+ domctl.u.psr_alloc.target = target;
+ domctl.u.psr_alloc.data = data;
return do_domctl(xch, &domctl);
}
@@ -294,31 +294,31 @@ int xc_psr_cat_get_domain_data(xc_interface *xch, uint32_t domid,
switch ( type )
{
case XC_PSR_CAT_L3_CBM:
- cmd = XEN_DOMCTL_PSR_CAT_OP_GET_L3_CBM;
+ cmd = XEN_DOMCTL_PSR_GET_L3_CBM;
break;
case XC_PSR_CAT_L3_CBM_CODE:
- cmd = XEN_DOMCTL_PSR_CAT_OP_GET_L3_CODE;
+ cmd = XEN_DOMCTL_PSR_GET_L3_CODE;
break;
case XC_PSR_CAT_L3_CBM_DATA:
- cmd = XEN_DOMCTL_PSR_CAT_OP_GET_L3_DATA;
+ cmd = XEN_DOMCTL_PSR_GET_L3_DATA;
break;
case XC_PSR_CAT_L2_CBM:
- cmd = XEN_DOMCTL_PSR_CAT_OP_GET_L2_CBM;
+ cmd = XEN_DOMCTL_PSR_GET_L2_CBM;
break;
default:
errno = EINVAL;
return -1;
}
- domctl.cmd = XEN_DOMCTL_psr_cat_op;
+ domctl.cmd = XEN_DOMCTL_psr_alloc;
domctl.domain = (domid_t)domid;
- domctl.u.psr_cat_op.cmd = cmd;
- domctl.u.psr_cat_op.target = target;
+ domctl.u.psr_alloc.cmd = cmd;
+ domctl.u.psr_alloc.target = target;
rc = do_domctl(xch, &domctl);
if ( !rc )
- *data = domctl.u.psr_cat_op.data;
+ *data = domctl.u.psr_alloc.data;
return rc;
}
@@ -329,30 +329,30 @@ int xc_psr_cat_get_info(xc_interface *xch, uint32_t socket, unsigned int lvl,
int rc = -1;
DECLARE_SYSCTL;
- sysctl.cmd = XEN_SYSCTL_psr_cat_op;
- sysctl.u.psr_cat_op.target = socket;
+ sysctl.cmd = XEN_SYSCTL_psr_alloc;
+ sysctl.u.psr_alloc.target = socket;
switch ( lvl )
{
case 2:
- sysctl.u.psr_cat_op.cmd = XEN_SYSCTL_PSR_CAT_get_l2_info;
+ sysctl.u.psr_alloc.cmd = XEN_SYSCTL_PSR_get_l2_info;
rc = xc_sysctl(xch, &sysctl);
if ( !rc )
{
- *cos_max = sysctl.u.psr_cat_op.u.cat_info.cos_max;
- *cbm_len = sysctl.u.psr_cat_op.u.cat_info.cbm_len;
+ *cos_max = sysctl.u.psr_alloc.u.cat_info.cos_max;
+ *cbm_len = sysctl.u.psr_alloc.u.cat_info.cbm_len;
*cdp_enabled = false;
}
break;
case 3:
- sysctl.u.psr_cat_op.cmd = XEN_SYSCTL_PSR_CAT_get_l3_info;
+ sysctl.u.psr_alloc.cmd = XEN_SYSCTL_PSR_get_l3_info;
rc = xc_sysctl(xch, &sysctl);
if ( !rc )
{
- *cos_max = sysctl.u.psr_cat_op.u.cat_info.cos_max;
- *cbm_len = sysctl.u.psr_cat_op.u.cat_info.cbm_len;
- *cdp_enabled = sysctl.u.psr_cat_op.u.cat_info.flags &
- XEN_SYSCTL_PSR_CAT_L3_CDP;
+ *cos_max = sysctl.u.psr_alloc.u.cat_info.cos_max;
+ *cbm_len = sysctl.u.psr_alloc.u.cat_info.cbm_len;
+ *cdp_enabled = sysctl.u.psr_alloc.u.cat_info.flags &
+ XEN_SYSCTL_PSR_L3_CDP;
}
break;
default:
diff --git a/xen/arch/x86/domctl.c b/xen/arch/x86/domctl.c
index 127c84e..b9afdf3 100644
--- a/xen/arch/x86/domctl.c
+++ b/xen/arch/x86/domctl.c
@@ -1439,60 +1439,60 @@ long arch_do_domctl(
}
break;
- case XEN_DOMCTL_psr_cat_op:
- switch ( domctl->u.psr_cat_op.cmd )
+ case XEN_DOMCTL_psr_alloc:
+ switch ( domctl->u.psr_alloc.cmd )
{
uint32_t val32;
- case XEN_DOMCTL_PSR_CAT_OP_SET_L3_CBM:
- ret = psr_set_val(d, domctl->u.psr_cat_op.target,
- domctl->u.psr_cat_op.data,
+ case XEN_DOMCTL_PSR_SET_L3_CBM:
+ ret = psr_set_val(d, domctl->u.psr_alloc.target,
+ domctl->u.psr_alloc.data,
PSR_CBM_TYPE_L3);
break;
- case XEN_DOMCTL_PSR_CAT_OP_SET_L3_CODE:
- ret = psr_set_val(d, domctl->u.psr_cat_op.target,
- domctl->u.psr_cat_op.data,
+ case XEN_DOMCTL_PSR_SET_L3_CODE:
+ ret = psr_set_val(d, domctl->u.psr_alloc.target,
+ domctl->u.psr_alloc.data,
PSR_CBM_TYPE_L3_CODE);
break;
- case XEN_DOMCTL_PSR_CAT_OP_SET_L3_DATA:
- ret = psr_set_val(d, domctl->u.psr_cat_op.target,
- domctl->u.psr_cat_op.data,
+ case XEN_DOMCTL_PSR_SET_L3_DATA:
+ ret = psr_set_val(d, domctl->u.psr_alloc.target,
+ domctl->u.psr_alloc.data,
PSR_CBM_TYPE_L3_DATA);
break;
- case XEN_DOMCTL_PSR_CAT_OP_SET_L2_CBM:
- ret = psr_set_val(d, domctl->u.psr_cat_op.target,
- domctl->u.psr_cat_op.data,
+ case XEN_DOMCTL_PSR_SET_L2_CBM:
+ ret = psr_set_val(d, domctl->u.psr_alloc.target,
+ domctl->u.psr_alloc.data,
PSR_CBM_TYPE_L2);
break;
- case XEN_DOMCTL_PSR_CAT_OP_GET_L3_CBM:
- ret = psr_get_val(d, domctl->u.psr_cat_op.target,
+ case XEN_DOMCTL_PSR_GET_L3_CBM:
+ ret = psr_get_val(d, domctl->u.psr_alloc.target,
&val32, PSR_CBM_TYPE_L3);
- domctl->u.psr_cat_op.data = val32;
+ domctl->u.psr_alloc.data = val32;
copyback = true;
break;
- case XEN_DOMCTL_PSR_CAT_OP_GET_L3_CODE:
- ret = psr_get_val(d, domctl->u.psr_cat_op.target,
+ case XEN_DOMCTL_PSR_GET_L3_CODE:
+ ret = psr_get_val(d, domctl->u.psr_alloc.target,
&val32, PSR_CBM_TYPE_L3_CODE);
- domctl->u.psr_cat_op.data = val32;
+ domctl->u.psr_alloc.data = val32;
copyback = true;
break;
- case XEN_DOMCTL_PSR_CAT_OP_GET_L3_DATA:
- ret = psr_get_val(d, domctl->u.psr_cat_op.target,
+ case XEN_DOMCTL_PSR_GET_L3_DATA:
+ ret = psr_get_val(d, domctl->u.psr_alloc.target,
&val32, PSR_CBM_TYPE_L3_DATA);
- domctl->u.psr_cat_op.data = val32;
+ domctl->u.psr_alloc.data = val32;
copyback = true;
break;
- case XEN_DOMCTL_PSR_CAT_OP_GET_L2_CBM:
- ret = psr_get_val(d, domctl->u.psr_cat_op.target,
+ case XEN_DOMCTL_PSR_GET_L2_CBM:
+ ret = psr_get_val(d, domctl->u.psr_alloc.target,
&val32, PSR_CBM_TYPE_L2);
- domctl->u.psr_cat_op.data = val32;
+ domctl->u.psr_alloc.data = val32;
copyback = true;
break;
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index daa2aeb..c851511 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -382,7 +382,7 @@ static bool l3_cdp_get_feat_info(const struct feat_node *feat,
if ( !cat_get_feat_info(feat, data, array_len) )
return false;
- data[PSR_INFO_IDX_CAT_FLAG] |= XEN_SYSCTL_PSR_CAT_L3_CDP;
+ data[PSR_INFO_IDX_CAT_FLAG] |= XEN_SYSCTL_PSR_L3_CDP;
return true;
}
diff --git a/xen/arch/x86/sysctl.c b/xen/arch/x86/sysctl.c
index c3fdae8..e44d8ad 100644
--- a/xen/arch/x86/sysctl.c
+++ b/xen/arch/x86/sysctl.c
@@ -171,45 +171,45 @@ long arch_do_sysctl(
break;
- case XEN_SYSCTL_psr_cat_op:
- switch ( sysctl->u.psr_cat_op.cmd )
+ case XEN_SYSCTL_psr_alloc:
+ switch ( sysctl->u.psr_alloc.cmd )
{
uint32_t data[PSR_INFO_ARRAY_SIZE];
- case XEN_SYSCTL_PSR_CAT_get_l3_info:
+ case XEN_SYSCTL_PSR_get_l3_info:
{
- ret = psr_get_info(sysctl->u.psr_cat_op.target,
+ ret = psr_get_info(sysctl->u.psr_alloc.target,
PSR_CBM_TYPE_L3, data, ARRAY_SIZE(data));
if ( ret )
break;
- sysctl->u.psr_cat_op.u.cat_info.cos_max =
+ sysctl->u.psr_alloc.u.cat_info.cos_max =
data[PSR_INFO_IDX_COS_MAX];
- sysctl->u.psr_cat_op.u.cat_info.cbm_len =
+ sysctl->u.psr_alloc.u.cat_info.cbm_len =
data[PSR_INFO_IDX_CAT_CBM_LEN];
- sysctl->u.psr_cat_op.u.cat_info.flags =
+ sysctl->u.psr_alloc.u.cat_info.flags =
data[PSR_INFO_IDX_CAT_FLAG];
- if ( __copy_field_to_guest(u_sysctl, sysctl, u.psr_cat_op) )
+ if ( __copy_field_to_guest(u_sysctl, sysctl, u.psr_alloc) )
ret = -EFAULT;
break;
}
- case XEN_SYSCTL_PSR_CAT_get_l2_info:
+ case XEN_SYSCTL_PSR_get_l2_info:
{
- ret = psr_get_info(sysctl->u.psr_cat_op.target,
+ ret = psr_get_info(sysctl->u.psr_alloc.target,
PSR_CBM_TYPE_L2, data, ARRAY_SIZE(data));
if ( ret )
break;
- sysctl->u.psr_cat_op.u.cat_info.cos_max =
+ sysctl->u.psr_alloc.u.cat_info.cos_max =
data[PSR_INFO_IDX_COS_MAX];
- sysctl->u.psr_cat_op.u.cat_info.cbm_len =
+ sysctl->u.psr_alloc.u.cat_info.cbm_len =
data[PSR_INFO_IDX_CAT_CBM_LEN];
- sysctl->u.psr_cat_op.u.cat_info.flags =
+ sysctl->u.psr_alloc.u.cat_info.flags =
data[PSR_INFO_IDX_CAT_FLAG];
- if ( __copy_field_to_guest(u_sysctl, sysctl, u.psr_cat_op) )
+ if ( __copy_field_to_guest(u_sysctl, sysctl, u.psr_alloc) )
ret = -EFAULT;
break;
}
diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
index 50ff58f..0c6f6ae 100644
--- a/xen/include/public/domctl.h
+++ b/xen/include/public/domctl.h
@@ -37,7 +37,7 @@
#include "hvm/save.h"
#include "memory.h"
-#define XEN_DOMCTL_INTERFACE_VERSION 0x0000000e
+#define XEN_DOMCTL_INTERFACE_VERSION 0x0000000f
/*
* NB. xen_domctl.domain is an IN/OUT parameter for this operation.
@@ -1147,21 +1147,21 @@ struct xen_domctl_monitor_op {
typedef struct xen_domctl_monitor_op xen_domctl_monitor_op_t;
DEFINE_XEN_GUEST_HANDLE(xen_domctl_monitor_op_t);
-struct xen_domctl_psr_cat_op {
-#define XEN_DOMCTL_PSR_CAT_OP_SET_L3_CBM 0
-#define XEN_DOMCTL_PSR_CAT_OP_GET_L3_CBM 1
-#define XEN_DOMCTL_PSR_CAT_OP_SET_L3_CODE 2
-#define XEN_DOMCTL_PSR_CAT_OP_SET_L3_DATA 3
-#define XEN_DOMCTL_PSR_CAT_OP_GET_L3_CODE 4
-#define XEN_DOMCTL_PSR_CAT_OP_GET_L3_DATA 5
-#define XEN_DOMCTL_PSR_CAT_OP_SET_L2_CBM 6
-#define XEN_DOMCTL_PSR_CAT_OP_GET_L2_CBM 7
- uint32_t cmd; /* IN: XEN_DOMCTL_PSR_CAT_OP_* */
+struct xen_domctl_psr_alloc {
+#define XEN_DOMCTL_PSR_SET_L3_CBM 0
+#define XEN_DOMCTL_PSR_GET_L3_CBM 1
+#define XEN_DOMCTL_PSR_SET_L3_CODE 2
+#define XEN_DOMCTL_PSR_SET_L3_DATA 3
+#define XEN_DOMCTL_PSR_GET_L3_CODE 4
+#define XEN_DOMCTL_PSR_GET_L3_DATA 5
+#define XEN_DOMCTL_PSR_SET_L2_CBM 6
+#define XEN_DOMCTL_PSR_GET_L2_CBM 7
+ uint32_t cmd; /* IN: XEN_DOMCTL_PSR_* */
uint32_t target; /* IN */
uint64_t data; /* IN/OUT */
};
-typedef struct xen_domctl_psr_cat_op xen_domctl_psr_cat_op_t;
-DEFINE_XEN_GUEST_HANDLE(xen_domctl_psr_cat_op_t);
+typedef struct xen_domctl_psr_alloc xen_domctl_psr_alloc_t;
+DEFINE_XEN_GUEST_HANDLE(xen_domctl_psr_alloc_t);
struct xen_domctl {
uint32_t cmd;
@@ -1238,7 +1238,7 @@ struct xen_domctl {
#define XEN_DOMCTL_setvnumainfo 74
#define XEN_DOMCTL_psr_cmt_op 75
#define XEN_DOMCTL_monitor_op 77
-#define XEN_DOMCTL_psr_cat_op 78
+#define XEN_DOMCTL_psr_alloc 78
#define XEN_DOMCTL_soft_reset 79
#define XEN_DOMCTL_gdbsx_guestmemio 1000
#define XEN_DOMCTL_gdbsx_pausevcpu 1001
@@ -1301,7 +1301,7 @@ struct xen_domctl {
struct xen_domctl_vnuma vnuma;
struct xen_domctl_psr_cmt_op psr_cmt_op;
struct xen_domctl_monitor_op monitor_op;
- struct xen_domctl_psr_cat_op psr_cat_op;
+ struct xen_domctl_psr_alloc psr_alloc;
uint8_t pad[128];
} u;
};
diff --git a/xen/include/public/sysctl.h b/xen/include/public/sysctl.h
index 7830b98..9cb7e19 100644
--- a/xen/include/public/sysctl.h
+++ b/xen/include/public/sysctl.h
@@ -36,7 +36,7 @@
#include "physdev.h"
#include "tmem.h"
-#define XEN_SYSCTL_INTERFACE_VERSION 0x0000000F
+#define XEN_SYSCTL_INTERFACE_VERSION 0x00000010
/*
* Read console content from Xen buffer ring.
@@ -743,22 +743,22 @@ struct xen_sysctl_pcitopoinfo {
typedef struct xen_sysctl_pcitopoinfo xen_sysctl_pcitopoinfo_t;
DEFINE_XEN_GUEST_HANDLE(xen_sysctl_pcitopoinfo_t);
-#define XEN_SYSCTL_PSR_CAT_get_l3_info 0
-#define XEN_SYSCTL_PSR_CAT_get_l2_info 1
-struct xen_sysctl_psr_cat_op {
- uint32_t cmd; /* IN: XEN_SYSCTL_PSR_CAT_* */
+#define XEN_SYSCTL_PSR_get_l3_info 0
+#define XEN_SYSCTL_PSR_get_l2_info 1
+struct xen_sysctl_psr_alloc {
+ uint32_t cmd; /* IN: XEN_SYSCTL_PSR_* */
uint32_t target; /* IN */
union {
struct {
uint32_t cbm_len; /* OUT: CBM length */
uint32_t cos_max; /* OUT: Maximum COS */
-#define XEN_SYSCTL_PSR_CAT_L3_CDP (1u << 0)
+#define XEN_SYSCTL_PSR_L3_CDP (1u << 0)
uint32_t flags; /* OUT: CAT flags */
} cat_info;
} u;
};
-typedef struct xen_sysctl_psr_cat_op xen_sysctl_psr_cat_op_t;
-DEFINE_XEN_GUEST_HANDLE(xen_sysctl_psr_cat_op_t);
+typedef struct xen_sysctl_psr_alloc xen_sysctl_psr_alloc_t;
+DEFINE_XEN_GUEST_HANDLE(xen_sysctl_psr_alloc_t);
#define XEN_SYSCTL_TMEM_OP_ALL_CLIENTS 0xFFFFU
@@ -1137,7 +1137,7 @@ struct xen_sysctl {
#define XEN_SYSCTL_gcov_op 20
#define XEN_SYSCTL_psr_cmt_op 21
#define XEN_SYSCTL_pcitopoinfo 22
-#define XEN_SYSCTL_psr_cat_op 23
+#define XEN_SYSCTL_psr_alloc 23
#define XEN_SYSCTL_tmem_op 24
#define XEN_SYSCTL_get_cpu_levelling_caps 25
#define XEN_SYSCTL_get_cpu_featureset 26
@@ -1166,7 +1166,7 @@ struct xen_sysctl {
struct xen_sysctl_scheduler_op scheduler_op;
struct xen_sysctl_gcov_op gcov_op;
struct xen_sysctl_psr_cmt_op psr_cmt_op;
- struct xen_sysctl_psr_cat_op psr_cat_op;
+ struct xen_sysctl_psr_alloc psr_alloc;
struct xen_sysctl_tmem_op tmem_op;
struct xen_sysctl_cpu_levelling_caps cpu_levelling_caps;
struct xen_sysctl_cpu_featureset cpu_featureset;
diff --git a/xen/xsm/flask/hooks.c b/xen/xsm/flask/hooks.c
index 56dc5b0..e5387c6 100644
--- a/xen/xsm/flask/hooks.c
+++ b/xen/xsm/flask/hooks.c
@@ -743,8 +743,8 @@ static int flask_domctl(struct domain *d, int cmd)
case XEN_DOMCTL_psr_cmt_op:
return current_has_perm(d, SECCLASS_DOMAIN2, DOMAIN2__PSR_CMT_OP);
- case XEN_DOMCTL_psr_cat_op:
- return current_has_perm(d, SECCLASS_DOMAIN2, DOMAIN2__PSR_CAT_OP);
+ case XEN_DOMCTL_psr_alloc:
+ return current_has_perm(d, SECCLASS_DOMAIN2, DOMAIN2__PSR_ALLOC);
case XEN_DOMCTL_soft_reset:
return current_has_perm(d, SECCLASS_DOMAIN2, DOMAIN2__SOFT_RESET);
@@ -807,9 +807,9 @@ static int flask_sysctl(int cmd)
case XEN_SYSCTL_psr_cmt_op:
return avc_current_has_perm(SECINITSID_XEN, SECCLASS_XEN2,
XEN2__PSR_CMT_OP, NULL);
- case XEN_SYSCTL_psr_cat_op:
+ case XEN_SYSCTL_psr_alloc:
return avc_current_has_perm(SECINITSID_XEN, SECCLASS_XEN2,
- XEN2__PSR_CAT_OP, NULL);
+ XEN2__PSR_ALLOC, NULL);
case XEN_SYSCTL_tmem_op:
return domain_has_xen(current->domain, XEN__TMEM_CONTROL);
diff --git a/xen/xsm/flask/policy/access_vectors b/xen/xsm/flask/policy/access_vectors
index da9f3df..df5f372 100644
--- a/xen/xsm/flask/policy/access_vectors
+++ b/xen/xsm/flask/policy/access_vectors
@@ -85,8 +85,8 @@ class xen2
resource_op
# XEN_SYSCTL_psr_cmt_op
psr_cmt_op
-# XEN_SYSCTL_psr_cat_op
- psr_cat_op
+# XEN_SYSCTL_psr_alloc
+ psr_alloc
# XENPF_get_symbol
get_symbol
# PMU control
@@ -246,8 +246,8 @@ class domain2
mem_paging
# XENMEM_sharing_op
mem_sharing
-# XEN_DOMCTL_psr_cat_op
- psr_cat_op
+# XEN_DOMCTL_psr_alloc
+ psr_alloc
}
# Similar to class domain, but primarily contains domctls related to HVM domains
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v4 03/15] x86: rename 'cbm_type' to 'psr_type' to make it general
2017-09-23 9:48 [PATCH v4 00/15] Enable Memory Bandwidth Allocation in Xen Yi Sun
2017-09-23 9:48 ` [PATCH v4 01/15] docs: create Memory Bandwidth Allocation (MBA) feature document Yi Sun
2017-09-23 9:48 ` [PATCH v4 02/15] Rename PSR sysctl/domctl interfaces and xsm policy to make them be general Yi Sun
@ 2017-09-23 9:48 ` Yi Sun
2017-09-28 10:25 ` Jan Beulich
2017-09-23 9:48 ` [PATCH v4 04/15] x86: implement data structure and CPU init flow for MBA Yi Sun
` (13 subsequent siblings)
16 siblings, 1 reply; 64+ messages in thread
From: Yi Sun @ 2017-09-23 9:48 UTC (permalink / raw)
To: xen-devel
Cc: Wei Liu, Yi Sun, Andrew Cooper, Jan Beulich, Chao Peng,
Roger Pau Monné
This patch renames 'cbm_type' to 'psr_type' to generalize it.
Then, we can reuse this for all psr allocation features.
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Roger Pau Monn? <roger.pau@citrix.com>
---
CC: Jan Beulich <jbeulich@suse.com>
CC: Andrew Cooper <andrew.cooper3@citrix.com>
CC: Wei Liu <wei.liu2@citrix.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Chao Peng <chao.p.peng@linux.intel.com>
v4:
- fix words in commit message.
(suggested by Roger Pau Monné)
v3:
- replace 'psr_val_type' to 'psr_type' and remove '_VAL' from the enum
items.
(suggested by Roger Pau Monné)
v2:
- replace 'PSR_VAL_TYPE_{L3, L2}' to 'PSR_VAL_TYPE_{L3, L2}_CBM'.
(suggested by Chao Peng)
---
xen/arch/x86/domctl.c | 16 ++++++------
xen/arch/x86/psr.c | 62 +++++++++++++++++++++++++----------------------
xen/arch/x86/sysctl.c | 4 +--
xen/include/asm-x86/psr.h | 18 +++++++-------
4 files changed, 52 insertions(+), 48 deletions(-)
diff --git a/xen/arch/x86/domctl.c b/xen/arch/x86/domctl.c
index b9afdf3..18cae88 100644
--- a/xen/arch/x86/domctl.c
+++ b/xen/arch/x86/domctl.c
@@ -1447,51 +1447,51 @@ long arch_do_domctl(
case XEN_DOMCTL_PSR_SET_L3_CBM:
ret = psr_set_val(d, domctl->u.psr_alloc.target,
domctl->u.psr_alloc.data,
- PSR_CBM_TYPE_L3);
+ PSR_TYPE_L3_CBM);
break;
case XEN_DOMCTL_PSR_SET_L3_CODE:
ret = psr_set_val(d, domctl->u.psr_alloc.target,
domctl->u.psr_alloc.data,
- PSR_CBM_TYPE_L3_CODE);
+ PSR_TYPE_L3_CODE);
break;
case XEN_DOMCTL_PSR_SET_L3_DATA:
ret = psr_set_val(d, domctl->u.psr_alloc.target,
domctl->u.psr_alloc.data,
- PSR_CBM_TYPE_L3_DATA);
+ PSR_TYPE_L3_DATA);
break;
case XEN_DOMCTL_PSR_SET_L2_CBM:
ret = psr_set_val(d, domctl->u.psr_alloc.target,
domctl->u.psr_alloc.data,
- PSR_CBM_TYPE_L2);
+ PSR_TYPE_L2_CBM);
break;
case XEN_DOMCTL_PSR_GET_L3_CBM:
ret = psr_get_val(d, domctl->u.psr_alloc.target,
- &val32, PSR_CBM_TYPE_L3);
+ &val32, PSR_TYPE_L3_CBM);
domctl->u.psr_alloc.data = val32;
copyback = true;
break;
case XEN_DOMCTL_PSR_GET_L3_CODE:
ret = psr_get_val(d, domctl->u.psr_alloc.target,
- &val32, PSR_CBM_TYPE_L3_CODE);
+ &val32, PSR_TYPE_L3_CODE);
domctl->u.psr_alloc.data = val32;
copyback = true;
break;
case XEN_DOMCTL_PSR_GET_L3_DATA:
ret = psr_get_val(d, domctl->u.psr_alloc.target,
- &val32, PSR_CBM_TYPE_L3_DATA);
+ &val32, PSR_TYPE_L3_DATA);
domctl->u.psr_alloc.data = val32;
copyback = true;
break;
case XEN_DOMCTL_PSR_GET_L2_CBM:
ret = psr_get_val(d, domctl->u.psr_alloc.target,
- &val32, PSR_CBM_TYPE_L2);
+ &val32, PSR_TYPE_L2_CBM);
domctl->u.psr_alloc.data = val32;
copyback = true;
break;
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index c851511..aa5a60f 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -100,24 +100,24 @@ static const struct feat_props {
unsigned int cos_num;
/*
- * An array to save all 'enum cbm_type' values of the feature. It is
+ * An array to save all 'enum psr_type' values of the feature. It is
* used with cos_num together to get/write a feature's COS registers
* values one by one.
*/
- enum cbm_type type[MAX_COS_NUM];
+ enum psr_type type[MAX_COS_NUM];
/*
* alt_type is 'alternative type'. When this 'alt_type' is input, the
* feature does some special operations.
*/
- enum cbm_type alt_type;
+ enum psr_type alt_type;
/* get_feat_info is used to return feature HW info through sysctl. */
bool (*get_feat_info)(const struct feat_node *feat,
uint32_t data[], unsigned int array_len);
/* write_msr is used to write out feature MSR register. */
- void (*write_msr)(unsigned int cos, uint32_t val, enum cbm_type type);
+ void (*write_msr)(unsigned int cos, uint32_t val, enum psr_type type);
} *feat_props[FEAT_TYPE_NUM];
/*
@@ -215,13 +215,13 @@ static void free_socket_resources(unsigned int socket)
bitmap_zero(info->dom_set, DOMID_IDLE + 1);
}
-static enum psr_feat_type psr_cbm_type_to_feat_type(enum cbm_type type)
+static enum psr_feat_type psr_type_to_feat_type(enum psr_type type)
{
enum psr_feat_type feat_type = FEAT_TYPE_UNKNOWN;
switch ( type )
{
- case PSR_CBM_TYPE_L3:
+ case PSR_TYPE_L3_CBM:
feat_type = FEAT_TYPE_L3_CAT;
/*
@@ -233,12 +233,12 @@ static enum psr_feat_type psr_cbm_type_to_feat_type(enum cbm_type type)
break;
- case PSR_CBM_TYPE_L3_DATA:
- case PSR_CBM_TYPE_L3_CODE:
+ case PSR_TYPE_L3_DATA:
+ case PSR_TYPE_L3_CODE:
feat_type = FEAT_TYPE_L3_CDP;
break;
- case PSR_CBM_TYPE_L2:
+ case PSR_TYPE_L2_CBM:
feat_type = FEAT_TYPE_L2_CAT;
break;
@@ -362,15 +362,16 @@ static bool cat_get_feat_info(const struct feat_node *feat,
}
/* L3 CAT props */
-static void l3_cat_write_msr(unsigned int cos, uint32_t val, enum cbm_type type)
+static void l3_cat_write_msr(unsigned int cos, uint32_t val,
+ enum psr_type type)
{
wrmsrl(MSR_IA32_PSR_L3_MASK(cos), val);
}
static const struct feat_props l3_cat_props = {
.cos_num = 1,
- .type[0] = PSR_CBM_TYPE_L3,
- .alt_type = PSR_CBM_TYPE_UNKNOWN,
+ .type[0] = PSR_TYPE_L3_CBM,
+ .alt_type = PSR_TYPE_UNKNOWN,
.get_feat_info = cat_get_feat_info,
.write_msr = l3_cat_write_msr,
};
@@ -387,9 +388,10 @@ static bool l3_cdp_get_feat_info(const struct feat_node *feat,
return true;
}
-static void l3_cdp_write_msr(unsigned int cos, uint32_t val, enum cbm_type type)
+static void l3_cdp_write_msr(unsigned int cos, uint32_t val,
+ enum psr_type type)
{
- wrmsrl(((type == PSR_CBM_TYPE_L3_DATA) ?
+ wrmsrl(((type == PSR_TYPE_L3_DATA) ?
MSR_IA32_PSR_L3_MASK_DATA(cos) :
MSR_IA32_PSR_L3_MASK_CODE(cos)),
val);
@@ -397,23 +399,24 @@ static void l3_cdp_write_msr(unsigned int cos, uint32_t val, enum cbm_type type)
static const struct feat_props l3_cdp_props = {
.cos_num = 2,
- .type[0] = PSR_CBM_TYPE_L3_DATA,
- .type[1] = PSR_CBM_TYPE_L3_CODE,
- .alt_type = PSR_CBM_TYPE_L3,
+ .type[0] = PSR_TYPE_L3_DATA,
+ .type[1] = PSR_TYPE_L3_CODE,
+ .alt_type = PSR_TYPE_L3_CBM,
.get_feat_info = l3_cdp_get_feat_info,
.write_msr = l3_cdp_write_msr,
};
/* L2 CAT props */
-static void l2_cat_write_msr(unsigned int cos, uint32_t val, enum cbm_type type)
+static void l2_cat_write_msr(unsigned int cos, uint32_t val,
+ enum psr_type type)
{
wrmsrl(MSR_IA32_PSR_L2_MASK(cos), val);
}
static const struct feat_props l2_cat_props = {
.cos_num = 1,
- .type[0] = PSR_CBM_TYPE_L2,
- .alt_type = PSR_CBM_TYPE_UNKNOWN,
+ .type[0] = PSR_TYPE_L2_CBM,
+ .alt_type = PSR_TYPE_UNKNOWN,
.get_feat_info = cat_get_feat_info,
.write_msr = l2_cat_write_msr,
};
@@ -675,7 +678,7 @@ static struct psr_socket_info *get_socket_info(unsigned int socket)
return socket_info + socket;
}
-int psr_get_info(unsigned int socket, enum cbm_type type,
+int psr_get_info(unsigned int socket, enum psr_type type,
uint32_t data[], unsigned int array_len)
{
const struct psr_socket_info *info = get_socket_info(socket);
@@ -687,7 +690,7 @@ int psr_get_info(unsigned int socket, enum cbm_type type,
if ( IS_ERR(info) )
return PTR_ERR(info);
- feat_type = psr_cbm_type_to_feat_type(type);
+ feat_type = psr_type_to_feat_type(type);
if ( feat_type >= ARRAY_SIZE(info->features) )
return -ENOENT;
@@ -708,7 +711,7 @@ int psr_get_info(unsigned int socket, enum cbm_type type,
}
int psr_get_val(struct domain *d, unsigned int socket,
- uint32_t *val, enum cbm_type type)
+ uint32_t *val, enum psr_type type)
{
const struct psr_socket_info *info = get_socket_info(socket);
const struct feat_node *feat;
@@ -720,7 +723,7 @@ int psr_get_val(struct domain *d, unsigned int socket,
if ( IS_ERR(info) )
return PTR_ERR(info);
- feat_type = psr_cbm_type_to_feat_type(type);
+ feat_type = psr_type_to_feat_type(type);
if ( feat_type >= ARRAY_SIZE(info->features) )
return -ENOENT;
@@ -850,7 +853,7 @@ static int insert_val_into_array(uint32_t val[],
unsigned int array_len,
const struct psr_socket_info *info,
enum psr_feat_type feat_type,
- enum cbm_type type,
+ enum psr_type type,
uint32_t new_val)
{
const struct feat_node *feat;
@@ -886,8 +889,9 @@ static int insert_val_into_array(uint32_t val[],
/*
* Value setting position is same as feature array.
* For CDP, user may set both DATA and CODE to same value. For such case,
- * user input 'PSR_CBM_TYPE_L3' as type. The alternative type of CDP is same
- * as it. So we should set new_val to both of DATA and CODE under such case.
+ * user input 'PSR_TYPE_L3_CBM' as type. The alternative type of CDP is
+ * same as it. So we should set new_val to both of DATA and CODE under such
+ * case.
*/
for ( i = 0; i < props->cos_num; i++ )
{
@@ -1176,7 +1180,7 @@ static int write_psr_msrs(unsigned int socket, unsigned int cos,
}
int psr_set_val(struct domain *d, unsigned int socket,
- uint64_t new_val, enum cbm_type type)
+ uint64_t new_val, enum psr_type type)
{
unsigned int old_cos, array_len;
int cos, ret;
@@ -1192,7 +1196,7 @@ int psr_set_val(struct domain *d, unsigned int socket,
if ( new_val != val )
return -EINVAL;
- feat_type = psr_cbm_type_to_feat_type(type);
+ feat_type = psr_type_to_feat_type(type);
if ( feat_type >= ARRAY_SIZE(info->features) ||
!info->features[feat_type] )
return -ENOENT;
diff --git a/xen/arch/x86/sysctl.c b/xen/arch/x86/sysctl.c
index e44d8ad..9dee163 100644
--- a/xen/arch/x86/sysctl.c
+++ b/xen/arch/x86/sysctl.c
@@ -179,7 +179,7 @@ long arch_do_sysctl(
case XEN_SYSCTL_PSR_get_l3_info:
{
ret = psr_get_info(sysctl->u.psr_alloc.target,
- PSR_CBM_TYPE_L3, data, ARRAY_SIZE(data));
+ PSR_TYPE_L3_CBM, data, ARRAY_SIZE(data));
if ( ret )
break;
@@ -198,7 +198,7 @@ long arch_do_sysctl(
case XEN_SYSCTL_PSR_get_l2_info:
{
ret = psr_get_info(sysctl->u.psr_alloc.target,
- PSR_CBM_TYPE_L2, data, ARRAY_SIZE(data));
+ PSR_TYPE_L2_CBM, data, ARRAY_SIZE(data));
if ( ret )
break;
diff --git a/xen/include/asm-x86/psr.h b/xen/include/asm-x86/psr.h
index 18a42f3..cb3f067 100644
--- a/xen/include/asm-x86/psr.h
+++ b/xen/include/asm-x86/psr.h
@@ -53,12 +53,12 @@ struct psr_cmt {
struct psr_cmt_l3 l3;
};
-enum cbm_type {
- PSR_CBM_TYPE_L3,
- PSR_CBM_TYPE_L3_CODE,
- PSR_CBM_TYPE_L3_DATA,
- PSR_CBM_TYPE_L2,
- PSR_CBM_TYPE_UNKNOWN,
+enum psr_type {
+ PSR_TYPE_L3_CBM,
+ PSR_TYPE_L3_CODE,
+ PSR_TYPE_L3_DATA,
+ PSR_TYPE_L2_CBM,
+ PSR_TYPE_UNKNOWN,
};
extern struct psr_cmt *psr_cmt;
@@ -72,12 +72,12 @@ int psr_alloc_rmid(struct domain *d);
void psr_free_rmid(struct domain *d);
void psr_ctxt_switch_to(struct domain *d);
-int psr_get_info(unsigned int socket, enum cbm_type type,
+int psr_get_info(unsigned int socket, enum psr_type type,
uint32_t data[], unsigned int array_len);
int psr_get_val(struct domain *d, unsigned int socket,
- uint32_t *val, enum cbm_type type);
+ uint32_t *val, enum psr_type type);
int psr_set_val(struct domain *d, unsigned int socket,
- uint64_t val, enum cbm_type type);
+ uint64_t val, enum psr_type type);
void psr_domain_init(struct domain *d);
void psr_domain_free(struct domain *d);
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v4 04/15] x86: implement data structure and CPU init flow for MBA
2017-09-23 9:48 [PATCH v4 00/15] Enable Memory Bandwidth Allocation in Xen Yi Sun
` (2 preceding siblings ...)
2017-09-23 9:48 ` [PATCH v4 03/15] x86: rename 'cbm_type' to 'psr_type' to make it general Yi Sun
@ 2017-09-23 9:48 ` Yi Sun
2017-09-26 8:38 ` Roger Pau Monné
2017-09-28 11:00 ` Jan Beulich
2017-09-23 9:48 ` [PATCH v4 05/15] x86: implement get hw info " Yi Sun
` (12 subsequent siblings)
16 siblings, 2 replies; 64+ messages in thread
From: Yi Sun @ 2017-09-23 9:48 UTC (permalink / raw)
To: xen-devel
Cc: Wei Liu, Yi Sun, Andrew Cooper, Jan Beulich, Chao Peng,
Roger Pau Monné
This patch implements main data structures of MBA.
Like CAT features, MBA HW info has cos_max which means the max thrtl
register number, and thrtl_max which means the max throttle value
(delay value). It also has a flag to represent if the throttle
value is linear or not.
One thrtl register of MBA stores a throttle value for one or more
domains. The throttle value means the delay between L2 cache and next
cache level.
This patch also implements init flow for MBA and register stub
callback functions.
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
---
CC: Jan Beulich <jbeulich@suse.com>
CC: Andrew Cooper <andrew.cooper3@citrix.com>
CC: Wei Liu <wei.liu2@citrix.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Chao Peng <chao.p.peng@linux.intel.com>
v4:
- modify commit message.
(suggested by Roger Pau Monné)
- fix a comment.
(suggested by Roger Pau Monné)
- join two checks in a single if.
(suggested by Roger Pau Monné)
- remove redundant initialization of 'feat->cos_reg_val[0]'.
(suggested by Roger Pau Monné)
- change 'reg_b' to 'ebx'.
(suggested by Jan Beulich)
- change type of 'mba_init_feature' from 'int' to 'bool'.
(suggested by Roger Pau Monné)
- change type of 'cat_init_feature' from 'int' to 'bool'.
v3:
- replace 'psr_val_type' to 'psr_type'. Also, change 'PSR_VAL_TYPE_MBA' to
'PSR_TYPE_MBA_THRTL'.
(suggested by Roger Pau Monné)
- replace 'MBA_LINEAR' to 'MBA_LINEAR_MASK' to make the name more clear.
(suggested by Roger Pau Monné)
- replase 'cat_info'/'mba_info' to 'cat'/'mba' to make the names shorter.
(suggested by Roger Pau Monné)
- change type of 'linear' to 'bool'.
(suggested by Roger Pau Monné)
- make format string of printf in one line.
(suggested by Roger Pau Monné)
v2:
- modify commit message to replace 'cos register' to 'thrtl register' to
make it accurate.
(suggested by Chao Peng)
- restore the place of the sentence to assign value to 'feat->cbm_len'
because the MBA init flow is splitted out as a separate function in v1.
(suggested by Chao Peng)
- add comment to explain what the MBA thrtl defaul value '0' stands for.
(suggested by Chao Peng)
- check 'thrtl_max' under linear mode. It could not be euqal or larger than
100.
(suggested by Chao Peng)
v1:
- rebase codes onto L2 CAT v15.
- move comment to appropriate place.
(suggested by Chao Peng)
- implement 'mba_init_feature' and keep 'cat_init_feature'.
(suggested by Chao Peng)
- keep 'regs.b' into a local variable to avoid reading CPUID every time.
(suggested by Chao Peng)
---
xen/arch/x86/psr.c | 157 ++++++++++++++++++++++++++++++++--------
xen/include/asm-x86/msr-index.h | 1 +
xen/include/asm-x86/psr.h | 2 +
3 files changed, 131 insertions(+), 29 deletions(-)
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index aa5a60f..e2f8fd0 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -27,13 +27,16 @@
* - CMT Cache Monitoring Technology
* - COS/CLOS Class of Service. Also mean COS registers.
* - COS_MAX Max number of COS for the feature (minus 1)
+ * - MBA Memory Bandwidth Allocation
* - MSRs Machine Specific Registers
* - PSR Intel Platform Shared Resource
+ * - THRTL_MAX Max throttle value (delay value) of MBA
*/
#define PSR_CMT (1u << 0)
#define PSR_CAT (1u << 1)
#define PSR_CDP (1u << 2)
+#define PSR_MBA (1u << 3)
#define CAT_CBM_LEN_MASK 0x1f
#define CAT_COS_MAX_MASK 0xffff
@@ -60,10 +63,14 @@
*/
#define MAX_COS_NUM 2
+#define MBA_LINEAR_MASK (1u << 2)
+#define MBA_THRTL_MAX_MASK 0xfff
+
enum psr_feat_type {
FEAT_TYPE_L3_CAT,
FEAT_TYPE_L3_CDP,
FEAT_TYPE_L2_CAT,
+ FEAT_TYPE_MBA,
FEAT_TYPE_NUM,
FEAT_TYPE_UNKNOWN,
};
@@ -71,7 +78,6 @@ enum psr_feat_type {
/*
* This structure represents one feature.
* cos_max - The max COS registers number got through CPUID.
- * cbm_len - The length of CBM got through CPUID.
* cos_reg_val - Array to store the values of COS registers. One entry stores
* the value of one COS register.
* For L3 CAT and L2 CAT, one entry corresponds to one COS_ID.
@@ -80,9 +86,23 @@ enum psr_feat_type {
* cos_reg_val[1] (Code).
*/
struct feat_node {
- /* cos_max and cbm_len are common values for all features so far. */
+ /* cos_max is common among all features so far. */
unsigned int cos_max;
- unsigned int cbm_len;
+
+ /* Feature specific HW info. */
+ union {
+ struct {
+ /* The length of CBM got through CPUID. */
+ unsigned int cbm_len;
+ } cat;
+
+ struct {
+ /* The max throttling value got through CPUID. */
+ unsigned int thrtl_max;
+ bool linear;
+ } mba;
+ };
+
uint32_t cos_reg_val[MAX_COS_REG_CNT];
};
@@ -161,6 +181,7 @@ static DEFINE_PER_CPU(struct psr_assoc, psr_assoc);
*/
static struct feat_node *feat_l3;
static struct feat_node *feat_l2_cat;
+static struct feat_node *feat_mba;
/* Common functions */
#define cat_default_val(len) (0xffffffff >> (32 - (len)))
@@ -272,8 +293,8 @@ static bool psr_check_cbm(unsigned int cbm_len, unsigned long cbm)
return true;
}
-/* CAT common functions implementation. */
-static int cat_init_feature(const struct cpuid_leaf *regs,
+/* Implementation of allocation features' functions. */
+static bool cat_init_feature(const struct cpuid_leaf *regs,
struct feat_node *feat,
struct psr_socket_info *info,
enum psr_feat_type type)
@@ -286,25 +307,25 @@ static int cat_init_feature(const struct cpuid_leaf *regs,
/* No valid value so do not enable feature. */
if ( !regs->a || !regs->d )
- return -ENOENT;
+ return false;
- feat->cbm_len = (regs->a & CAT_CBM_LEN_MASK) + 1;
feat->cos_max = min(opt_cos_max, regs->d & CAT_COS_MAX_MASK);
+ feat->cat.cbm_len = (regs->a & CAT_CBM_LEN_MASK) + 1;
switch ( type )
{
case FEAT_TYPE_L3_CAT:
case FEAT_TYPE_L2_CAT:
if ( feat->cos_max < 1 )
- return -ENOENT;
+ return false;
/* We reserve cos=0 as default cbm (all bits within cbm_len are 1). */
- feat->cos_reg_val[0] = cat_default_val(feat->cbm_len);
+ feat->cos_reg_val[0] = cat_default_val(feat->cat.cbm_len);
wrmsrl((type == FEAT_TYPE_L3_CAT ?
MSR_IA32_PSR_L3_MASK(0) :
MSR_IA32_PSR_L2_MASK(0)),
- cat_default_val(feat->cbm_len));
+ cat_default_val(feat->cat.cbm_len));
break;
@@ -313,17 +334,19 @@ static int cat_init_feature(const struct cpuid_leaf *regs,
uint64_t val;
if ( feat->cos_max < 3 )
- return -ENOENT;
+ return false;
/* Cut half of cos_max when CDP is enabled. */
feat->cos_max = (feat->cos_max - 1) >> 1;
/* We reserve cos=0 as default cbm (all bits within cbm_len are 1). */
- get_cdp_code(feat, 0) = cat_default_val(feat->cbm_len);
- get_cdp_data(feat, 0) = cat_default_val(feat->cbm_len);
+ get_cdp_code(feat, 0) = cat_default_val(feat->cat.cbm_len);
+ get_cdp_data(feat, 0) = cat_default_val(feat->cat.cbm_len);
- wrmsrl(MSR_IA32_PSR_L3_MASK(0), cat_default_val(feat->cbm_len));
- wrmsrl(MSR_IA32_PSR_L3_MASK(1), cat_default_val(feat->cbm_len));
+ wrmsrl(MSR_IA32_PSR_L3_MASK(0),
+ cat_default_val(feat->cat.cbm_len));
+ wrmsrl(MSR_IA32_PSR_L3_MASK(1),
+ cat_default_val(feat->cat.cbm_len));
rdmsrl(MSR_IA32_PSR_L3_QOS_CFG, val);
wrmsrl(MSR_IA32_PSR_L3_QOS_CFG,
val | (1ull << PSR_L3_QOS_CDP_ENABLE_BIT));
@@ -332,20 +355,58 @@ static int cat_init_feature(const struct cpuid_leaf *regs,
}
default:
- return -ENOENT;
+ return false;
}
/* Add this feature into array. */
info->features[type] = feat;
if ( !opt_cpu_info )
- return 0;
+ return true;
printk(XENLOG_INFO "%s: enabled on socket %u, cos_max:%u, cbm_len:%u\n",
cat_feat_name[type], cpu_to_socket(smp_processor_id()),
- feat->cos_max, feat->cbm_len);
+ feat->cos_max, feat->cat.cbm_len);
- return 0;
+ return true;
+}
+
+static bool mba_init_feature(const struct cpuid_leaf *regs,
+ struct feat_node *feat,
+ struct psr_socket_info *info,
+ enum psr_feat_type type)
+{
+ /* No valid value so do not enable feature. */
+ if ( !regs->a || !regs->d || type != FEAT_TYPE_MBA )
+ return false;
+
+ feat->cos_max = min(opt_cos_max, regs->d & CAT_COS_MAX_MASK);
+ if ( feat->cos_max < 1 )
+ return false;
+
+ feat->mba.thrtl_max = (regs->a & MBA_THRTL_MAX_MASK) + 1;
+
+ if ( regs->c & MBA_LINEAR_MASK )
+ {
+ feat->mba.linear = true;
+
+ if ( feat->mba.thrtl_max >= 100 )
+ return false;
+ }
+
+ wrmsrl(MSR_IA32_PSR_MBA_MASK(0), 0);
+
+ /* Add this feature into array. */
+ info->features[type] = feat;
+
+ if ( !opt_cpu_info )
+ return true;
+
+ printk(XENLOG_INFO "MBA: enabled on socket %u, cos_max:%u, thrtl_max:%u, linear:%u.\n",
+ cpu_to_socket(smp_processor_id()),
+ feat->cos_max, feat->mba.thrtl_max, feat->mba.linear);
+
+ return true;
}
static bool cat_get_feat_info(const struct feat_node *feat,
@@ -355,7 +416,7 @@ static bool cat_get_feat_info(const struct feat_node *feat,
return false;
data[PSR_INFO_IDX_COS_MAX] = feat->cos_max;
- data[PSR_INFO_IDX_CAT_CBM_LEN] = feat->cbm_len;
+ data[PSR_INFO_IDX_CAT_CBM_LEN] = feat->cat.cbm_len;
data[PSR_INFO_IDX_CAT_FLAG] = 0;
return true;
@@ -421,6 +482,26 @@ static const struct feat_props l2_cat_props = {
.write_msr = l2_cat_write_msr,
};
+/* MBA props */
+static bool mba_get_feat_info(const struct feat_node *feat,
+ uint32_t data[], unsigned int array_len)
+{
+ return false;
+}
+
+static void mba_write_msr(unsigned int cos, uint32_t val,
+ enum psr_type type)
+{
+}
+
+static const struct feat_props mba_props = {
+ .cos_num = 1,
+ .type[0] = PSR_TYPE_MBA_THRTL,
+ .alt_type = PSR_TYPE_UNKNOWN,
+ .get_feat_info = mba_get_feat_info,
+ .write_msr = mba_write_msr,
+};
+
static bool __init parse_psr_bool(const char *s, const char *delim,
const char *ss, const char *feature,
unsigned int mask)
@@ -479,7 +560,8 @@ static int __init parse_psr_param(const char *s)
}
else if ( !parse_psr_bool(s, val_delim, ss, "cmt", PSR_CMT) &&
!parse_psr_bool(s, val_delim, ss, "cat", PSR_CAT) &&
- !parse_psr_bool(s, val_delim, ss, "cdp", PSR_CDP) )
+ !parse_psr_bool(s, val_delim, ss, "cdp", PSR_CDP) &&
+ !parse_psr_bool(s, val_delim, ss, "mba", PSR_MBA) )
rc = -EINVAL;
s = ss + 1;
@@ -883,7 +965,7 @@ static int insert_val_into_array(uint32_t val[],
if ( array_len < props->cos_num )
return -ENOSPC;
- if ( !psr_check_cbm(feat->cbm_len, new_val) )
+ if ( !psr_check_cbm(feat->cat.cbm_len, new_val) )
return -EINVAL;
/*
@@ -1401,6 +1483,10 @@ static int psr_cpu_prepare(void)
(feat_l2_cat = xzalloc(struct feat_node)) == NULL )
return -ENOMEM;
+ if ( feat_mba == NULL &&
+ (feat_mba = xzalloc(struct feat_node)) == NULL )
+ return -ENOMEM;
+
return 0;
}
@@ -1410,6 +1496,7 @@ static void psr_cpu_init(void)
unsigned int socket, cpu = smp_processor_id();
struct feat_node *feat;
struct cpuid_leaf regs;
+ uint32_t ebx;
if ( !psr_alloc_feat_enabled() || !boot_cpu_has(X86_FEATURE_PQE) )
goto assoc_init;
@@ -1428,7 +1515,8 @@ static void psr_cpu_init(void)
spin_lock_init(&info->ref_lock);
cpuid_count_leaf(PSR_CPUID_LEVEL_CAT, 0, ®s);
- if ( regs.b & PSR_RESOURCE_TYPE_L3 )
+ ebx = regs.b;
+ if ( ebx & PSR_RESOURCE_TYPE_L3 )
{
cpuid_count_leaf(PSR_CPUID_LEVEL_CAT, 1, ®s);
@@ -1436,32 +1524,43 @@ static void psr_cpu_init(void)
feat_l3 = NULL;
if ( (regs.c & PSR_CAT_CDP_CAPABILITY) && (opt_psr & PSR_CDP) &&
- !cat_init_feature(®s, feat, info, FEAT_TYPE_L3_CDP) )
+ cat_init_feature(®s, feat, info, FEAT_TYPE_L3_CDP) )
feat_props[FEAT_TYPE_L3_CDP] = &l3_cdp_props;
/* If CDP init fails, try to work as L3 CAT. */
if ( !feat_props[FEAT_TYPE_L3_CDP] )
{
- if ( !cat_init_feature(®s, feat, info, FEAT_TYPE_L3_CAT) )
+ if ( cat_init_feature(®s, feat, info, FEAT_TYPE_L3_CAT) )
feat_props[FEAT_TYPE_L3_CAT] = &l3_cat_props;
else
feat_l3 = feat;
}
}
- cpuid_count_leaf(PSR_CPUID_LEVEL_CAT, 0, ®s);
- if ( regs.b & PSR_RESOURCE_TYPE_L2 )
+ if ( ebx & PSR_RESOURCE_TYPE_L2 )
{
cpuid_count_leaf(PSR_CPUID_LEVEL_CAT, 2, ®s);
feat = feat_l2_cat;
feat_l2_cat = NULL;
- if ( !cat_init_feature(®s, feat, info, FEAT_TYPE_L2_CAT) )
+ if ( cat_init_feature(®s, feat, info, FEAT_TYPE_L2_CAT) )
feat_props[FEAT_TYPE_L2_CAT] = &l2_cat_props;
else
feat_l2_cat = feat;
}
+ if ( ebx & PSR_RESOURCE_TYPE_MBA )
+ {
+ cpuid_count_leaf(PSR_CPUID_LEVEL_CAT, 3, ®s);
+
+ feat = feat_mba;
+ feat_mba = NULL;
+ if ( mba_init_feature(®s, feat, info, FEAT_TYPE_MBA) )
+ feat_props[FEAT_TYPE_MBA] = &mba_props;
+ else
+ feat_mba = feat;
+ }
+
info->feat_init = true;
assoc_init:
@@ -1521,7 +1620,7 @@ static int __init psr_presmp_init(void)
if ( (opt_psr & PSR_CMT) && opt_rmid_max )
init_psr_cmt(opt_rmid_max);
- if ( opt_psr & (PSR_CAT | PSR_CDP) )
+ if ( opt_psr & (PSR_CAT | PSR_CDP | PSR_MBA) )
init_psr();
if ( psr_cpu_prepare() )
diff --git a/xen/include/asm-x86/msr-index.h b/xen/include/asm-x86/msr-index.h
index 4e08de6..41f1677 100644
--- a/xen/include/asm-x86/msr-index.h
+++ b/xen/include/asm-x86/msr-index.h
@@ -348,6 +348,7 @@
#define MSR_IA32_PSR_L3_MASK_CODE(n) (0x00000c90 + (n) * 2 + 1)
#define MSR_IA32_PSR_L3_MASK_DATA(n) (0x00000c90 + (n) * 2)
#define MSR_IA32_PSR_L2_MASK(n) (0x00000d10 + (n))
+#define MSR_IA32_PSR_MBA_MASK(n) (0x00000d50 + (n))
/* Intel Model 6 */
#define MSR_P6_PERFCTR(n) (0x000000c1 + (n))
diff --git a/xen/include/asm-x86/psr.h b/xen/include/asm-x86/psr.h
index cb3f067..9d14264 100644
--- a/xen/include/asm-x86/psr.h
+++ b/xen/include/asm-x86/psr.h
@@ -24,6 +24,7 @@
/* Resource Type Enumeration */
#define PSR_RESOURCE_TYPE_L3 0x2
#define PSR_RESOURCE_TYPE_L2 0x4
+#define PSR_RESOURCE_TYPE_MBA 0x8
/* L3 Monitoring Features */
#define PSR_CMT_L3_OCCUPANCY 0x1
@@ -58,6 +59,7 @@ enum psr_type {
PSR_TYPE_L3_CODE,
PSR_TYPE_L3_DATA,
PSR_TYPE_L2_CBM,
+ PSR_TYPE_MBA_THRTL,
PSR_TYPE_UNKNOWN,
};
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v4 05/15] x86: implement get hw info flow for MBA
2017-09-23 9:48 [PATCH v4 00/15] Enable Memory Bandwidth Allocation in Xen Yi Sun
` (3 preceding siblings ...)
2017-09-23 9:48 ` [PATCH v4 04/15] x86: implement data structure and CPU init flow for MBA Yi Sun
@ 2017-09-23 9:48 ` Yi Sun
2017-09-26 8:46 ` Roger Pau Monné
2017-09-23 9:48 ` [PATCH v4 06/15] x86: implement get value interface " Yi Sun
` (11 subsequent siblings)
16 siblings, 1 reply; 64+ messages in thread
From: Yi Sun @ 2017-09-23 9:48 UTC (permalink / raw)
To: xen-devel
Cc: Wei Liu, Yi Sun, Andrew Cooper, Jan Beulich, Chao Peng,
Roger Pau Monné
This patch implements get HW info flow for MBA including its callback
function and sysctl interface.
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
---
CC: Jan Beulich <jbeulich@suse.com>
CC: Andrew Cooper <andrew.cooper3@citrix.com>
CC: Wei Liu <wei.liu2@citrix.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Chao Peng <chao.p.peng@linux.intel.com>
v4:
- remove 'ALLOC_' from macro names.
(suggested by Roger Pau Monné)
- initialize 'data[PSR_INFO_ARRAY_SIZE]' to 0 to prevent to leak stack data.
(suggested by Roger Pau Monné)
v3:
- replace 'PSR_VAL_TYPE_MBA' to 'PSR_TYPE_MBA_THRTL'.
(suggested by Roger Pau Monné)
v2:
- use 'XEN_SYSCTL_PSR_MBA_LINEAR' to set MBA feature HW info.
(suggested by Chao Peng)
v1:
- sort 'PSR_INFO_IDX_' macros as feature.
(suggested by Chao Peng)
- rename 'PSR_INFO_IDX_MBA_LINEAR' to 'PSR_INFO_IDX_MBA_FLAG'.
- rename 'linear' in 'struct mba_info' to 'flags' for future extension.
(suggested by Chao Peng)
---
xen/arch/x86/psr.c | 15 ++++++++++++++-
xen/arch/x86/sysctl.c | 21 ++++++++++++++++++++-
xen/include/asm-x86/psr.h | 2 ++
xen/include/public/sysctl.h | 8 ++++++++
4 files changed, 44 insertions(+), 2 deletions(-)
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index e2f8fd0..1d0a215 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -263,6 +263,10 @@ static enum psr_feat_type psr_type_to_feat_type(enum psr_type type)
feat_type = FEAT_TYPE_L2_CAT;
break;
+ case PSR_TYPE_MBA_THRTL:
+ feat_type = FEAT_TYPE_MBA;
+ break;
+
default:
ASSERT_UNREACHABLE();
}
@@ -486,7 +490,16 @@ static const struct feat_props l2_cat_props = {
static bool mba_get_feat_info(const struct feat_node *feat,
uint32_t data[], unsigned int array_len)
{
- return false;
+ if ( array_len != PSR_INFO_ARRAY_SIZE )
+ return false;
+
+ data[PSR_INFO_IDX_COS_MAX] = feat->cos_max;
+ data[PSR_INFO_IDX_MBA_THRTL_MAX] = feat->mba.thrtl_max;
+
+ if ( feat->mba.linear )
+ data[PSR_INFO_IDX_MBA_FLAG] |= XEN_SYSCTL_PSR_MBA_LINEAR;
+
+ return true;
}
static void mba_write_msr(unsigned int cos, uint32_t val,
diff --git a/xen/arch/x86/sysctl.c b/xen/arch/x86/sysctl.c
index 9dee163..b2f8efa 100644
--- a/xen/arch/x86/sysctl.c
+++ b/xen/arch/x86/sysctl.c
@@ -174,7 +174,7 @@ long arch_do_sysctl(
case XEN_SYSCTL_psr_alloc:
switch ( sysctl->u.psr_alloc.cmd )
{
- uint32_t data[PSR_INFO_ARRAY_SIZE];
+ uint32_t data[PSR_INFO_ARRAY_SIZE] = {0};
case XEN_SYSCTL_PSR_get_l3_info:
{
@@ -214,6 +214,25 @@ long arch_do_sysctl(
break;
}
+ case XEN_SYSCTL_PSR_get_mba_info:
+ {
+ ret = psr_get_info(sysctl->u.psr_alloc.target,
+ PSR_TYPE_MBA_THRTL, data, ARRAY_SIZE(data));
+ if ( ret )
+ break;
+
+ sysctl->u.psr_alloc.u.mba_info.cos_max =
+ data[PSR_INFO_IDX_COS_MAX];
+ sysctl->u.psr_alloc.u.mba_info.thrtl_max =
+ data[PSR_INFO_IDX_MBA_THRTL_MAX];
+ sysctl->u.psr_alloc.u.mba_info.flags =
+ data[PSR_INFO_IDX_MBA_FLAG];
+
+ if ( __copy_field_to_guest(u_sysctl, sysctl, u.psr_alloc) )
+ ret = -EFAULT;
+ break;
+ }
+
default:
ret = -EOPNOTSUPP;
break;
diff --git a/xen/include/asm-x86/psr.h b/xen/include/asm-x86/psr.h
index 9d14264..084ae97 100644
--- a/xen/include/asm-x86/psr.h
+++ b/xen/include/asm-x86/psr.h
@@ -39,6 +39,8 @@
#define PSR_INFO_IDX_COS_MAX 0
#define PSR_INFO_IDX_CAT_CBM_LEN 1
#define PSR_INFO_IDX_CAT_FLAG 2
+#define PSR_INFO_IDX_MBA_THRTL_MAX 1
+#define PSR_INFO_IDX_MBA_FLAG 2
#define PSR_INFO_ARRAY_SIZE 3
struct psr_cmt_l3 {
diff --git a/xen/include/public/sysctl.h b/xen/include/public/sysctl.h
index 9cb7e19..559de59 100644
--- a/xen/include/public/sysctl.h
+++ b/xen/include/public/sysctl.h
@@ -745,6 +745,7 @@ DEFINE_XEN_GUEST_HANDLE(xen_sysctl_pcitopoinfo_t);
#define XEN_SYSCTL_PSR_get_l3_info 0
#define XEN_SYSCTL_PSR_get_l2_info 1
+#define XEN_SYSCTL_PSR_get_mba_info 2
struct xen_sysctl_psr_alloc {
uint32_t cmd; /* IN: XEN_SYSCTL_PSR_* */
uint32_t target; /* IN */
@@ -755,6 +756,13 @@ struct xen_sysctl_psr_alloc {
#define XEN_SYSCTL_PSR_L3_CDP (1u << 0)
uint32_t flags; /* OUT: CAT flags */
} cat_info;
+
+ struct {
+ uint32_t thrtl_max; /* OUT: Maximum throttle */
+ uint32_t cos_max; /* OUT: Maximum COS */
+#define XEN_SYSCTL_PSR_MBA_LINEAR (1u << 0)
+ uint32_t flags; /* OUT: MBA flags */
+ } mba_info;
} u;
};
typedef struct xen_sysctl_psr_alloc xen_sysctl_psr_alloc_t;
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v4 06/15] x86: implement get value interface for MBA
2017-09-23 9:48 [PATCH v4 00/15] Enable Memory Bandwidth Allocation in Xen Yi Sun
` (4 preceding siblings ...)
2017-09-23 9:48 ` [PATCH v4 05/15] x86: implement get hw info " Yi Sun
@ 2017-09-23 9:48 ` Yi Sun
2017-09-28 11:12 ` Jan Beulich
2017-09-23 9:48 ` [PATCH v4 07/15] x86: implement set value flow " Yi Sun
` (10 subsequent siblings)
16 siblings, 1 reply; 64+ messages in thread
From: Yi Sun @ 2017-09-23 9:48 UTC (permalink / raw)
To: xen-devel
Cc: Wei Liu, Yi Sun, Andrew Cooper, Jan Beulich, Chao Peng,
Roger Pau Monné
This patch implements get value domctl interface for MBA.
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
---
CC: Jan Beulich <jbeulich@suse.com>
CC: Andrew Cooper <andrew.cooper3@citrix.com>
CC: Wei Liu <wei.liu2@citrix.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Chao Peng <chao.p.peng@linux.intel.com>
v4:
- remove 'ALLOC_' from macro names.
(suggested by Roger Pau Monné)
v3:
- change 'PSR_VAL_TYPE_MBA' to 'PSR_TYPE_MBA_THRTL'.
(suggested by Roger Pau Monné)
---
xen/arch/x86/domctl.c | 7 +++++++
xen/include/public/domctl.h | 1 +
2 files changed, 8 insertions(+)
diff --git a/xen/arch/x86/domctl.c b/xen/arch/x86/domctl.c
index 18cae88..22650d7 100644
--- a/xen/arch/x86/domctl.c
+++ b/xen/arch/x86/domctl.c
@@ -1496,6 +1496,13 @@ long arch_do_domctl(
copyback = true;
break;
+ case XEN_DOMCTL_PSR_GET_MBA_THRTL:
+ ret = psr_get_val(d, domctl->u.psr_alloc.target,
+ &val32, PSR_TYPE_MBA_THRTL);
+ domctl->u.psr_alloc.data = val32;
+ copyback = true;
+ break;
+
default:
ret = -EOPNOTSUPP;
break;
diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
index 0c6f6ae..ec6d2de 100644
--- a/xen/include/public/domctl.h
+++ b/xen/include/public/domctl.h
@@ -1156,6 +1156,7 @@ struct xen_domctl_psr_alloc {
#define XEN_DOMCTL_PSR_GET_L3_DATA 5
#define XEN_DOMCTL_PSR_SET_L2_CBM 6
#define XEN_DOMCTL_PSR_GET_L2_CBM 7
+#define XEN_DOMCTL_PSR_GET_MBA_THRTL 9
uint32_t cmd; /* IN: XEN_DOMCTL_PSR_* */
uint32_t target; /* IN */
uint64_t data; /* IN/OUT */
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v4 07/15] x86: implement set value flow for MBA
2017-09-23 9:48 [PATCH v4 00/15] Enable Memory Bandwidth Allocation in Xen Yi Sun
` (5 preceding siblings ...)
2017-09-23 9:48 ` [PATCH v4 06/15] x86: implement get value interface " Yi Sun
@ 2017-09-23 9:48 ` Yi Sun
2017-09-26 9:39 ` Roger Pau Monné
2017-09-28 11:36 ` Jan Beulich
2017-09-23 9:48 ` [PATCH v4 08/15] tools: create general interfaces to support psr allocation features Yi Sun
` (9 subsequent siblings)
16 siblings, 2 replies; 64+ messages in thread
From: Yi Sun @ 2017-09-23 9:48 UTC (permalink / raw)
To: xen-devel
Cc: Wei Liu, Yi Sun, Andrew Cooper, Jan Beulich, Chao Peng,
Roger Pau Monné
This patch implements set value flow for MBA including its callback
function and domctl interface.
It also changes the memebers in 'cos_write_info' to transfer the
feature array, feature properties array and value array. Then, we
can write all features values on the cos id into MSRs.
Because multiple features may co-exist, we need handle all features to write
values of them into a COS register with new COS ID. E.g:
1. L3 CAT and MBA co-exist.
2. Dom1 and Dom2 share a same COS ID (2). The L3 CAT CBM of Dom1 is 0x1ff,
the MBA Thrtle of Dom1 is 0xa.
3. User wants to change MBA Thrtl of Dom1 to be 0x14. Because COS ID 2 is
used by Dom2 too, we have to pick a new COS ID 3. The values of Dom1 on
COS ID 3 are all default values as below:
---------
| COS 3 |
---------
L3 CAT | 0x7ff |
---------
MBA | 0x0 |
---------
4. After setting, the L3 CAT CBM value of Dom1 should be kept and the new MBA
Thrtl is set. So, the values on COS ID 3 should be below.
---------
| COS 3 |
---------
L3 CAT | 0x1ff |
---------
MBA | 0x14 |
---------
So, we should write all features values into their MSRs. That requires the
feature array, feature properties array and value array are input.
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
---
CC: Jan Beulich <jbeulich@suse.com>
CC: Andrew Cooper <andrew.cooper3@citrix.com>
CC: Wei Liu <wei.liu2@citrix.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Chao Peng <chao.p.peng@linux.intel.com>
v4:
- remove 'ALLOC_' from macro names.
(suggested by Roger Pau Monné)
- join two checks into a single if.
(suggested by Roger Pau Monné)
- remove redundant local variable 'array_len'.
(suggested by Roger Pau Monné)
v3:
- modify commit message to make it clear.
(suggested by Roger Pau Monné)
- modify functionality of 'check_val' to make it simple to only check value.
Change the last parameter type from 'unsigned long *' to 'unsigned long'.
(suggested by Roger Pau Monné)
- call rdmsrl to get value just written into MSR for MBA. Because HW can
automatically change input value to what it wants.
(suggested by Roger Pau Monné)
- change type of 'write_msr' to 'uint32_t' to return the value actually
written into MSR. Then, change 'do_write_psr_msrs' to set the returned
value into 'cos_reg_val[]'
- move the declaration of 'j' into loop in 'do_write_psr_msrs'.
(suggested by Roger Pau Monné)
- change 'mba_info' to 'mba'.
(suggested by Roger Pau Monné)
- change 'cat_info' to 'cat'.
(suggested by Roger Pau Monné)
- rename 'psr_cat/PSR_CAT' to 'psr_alloc/PSR_ALLOC' and remove 'op/OP'
from name.
(suggested by Roger Pau Monné)
- change 'PSR_VAL_TYPE_MBA' to 'PSR_TYPE_MBA_THRTL'.
(suggested by Roger Pau Monné)
v2:
- remove linear mode 'thrtl_max' check in 'mba_check_thrtl' because it has
been checked in 'mba_init_feature'.
(suggested by Chao Peng)
- for non-linear mode, check if '*thrtl' is not 0 in 'mba_check_thrtl'. If
it is 0, we do not need to change it.
(suggested by Chao Peng)
- move comments to explain changes of 'cos_write_info' from psr.c to commit
message.
(suggested by Chao Peng)
---
xen/arch/x86/domctl.c | 6 ++
xen/arch/x86/psr.c | 144 ++++++++++++++++++++++++++------------------
xen/include/public/domctl.h | 1 +
3 files changed, 94 insertions(+), 57 deletions(-)
diff --git a/xen/arch/x86/domctl.c b/xen/arch/x86/domctl.c
index 22650d7..66c9cab 100644
--- a/xen/arch/x86/domctl.c
+++ b/xen/arch/x86/domctl.c
@@ -1468,6 +1468,12 @@ long arch_do_domctl(
PSR_TYPE_L2_CBM);
break;
+ case XEN_DOMCTL_PSR_SET_MBA_THRTL:
+ ret = psr_set_val(d, domctl->u.psr_alloc.target,
+ domctl->u.psr_alloc.data,
+ PSR_TYPE_MBA_THRTL);
+ break;
+
case XEN_DOMCTL_PSR_GET_L3_CBM:
ret = psr_get_val(d, domctl->u.psr_alloc.target,
&val32, PSR_TYPE_L3_CBM);
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index 1d0a215..c6d4e31 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -137,7 +137,10 @@ static const struct feat_props {
uint32_t data[], unsigned int array_len);
/* write_msr is used to write out feature MSR register. */
- void (*write_msr)(unsigned int cos, uint32_t val, enum psr_type type);
+ uint32_t (*write_msr)(unsigned int cos, uint32_t val, enum psr_type type);
+
+ /* check_val is used to check if input val fulfills SDM requirement. */
+ bool (*check_val)(const struct feat_node *feat, unsigned long val);
} *feat_props[FEAT_TYPE_NUM];
/*
@@ -274,29 +277,6 @@ static enum psr_feat_type psr_type_to_feat_type(enum psr_type type)
return feat_type;
}
-static bool psr_check_cbm(unsigned int cbm_len, unsigned long cbm)
-{
- unsigned int first_bit, zero_bit;
-
- /* Set bits should only in the range of [0, cbm_len]. */
- if ( cbm & (~0ul << cbm_len) )
- return false;
-
- /* At least one bit need to be set. */
- if ( cbm == 0 )
- return false;
-
- first_bit = find_first_bit(&cbm, cbm_len);
- zero_bit = find_next_zero_bit(&cbm, cbm_len, first_bit);
-
- /* Set bits should be contiguous. */
- if ( zero_bit < cbm_len &&
- find_next_bit(&cbm, cbm_len, zero_bit) < cbm_len )
- return false;
-
- return true;
-}
-
/* Implementation of allocation features' functions. */
static bool cat_init_feature(const struct cpuid_leaf *regs,
struct feat_node *feat,
@@ -426,11 +406,36 @@ static bool cat_get_feat_info(const struct feat_node *feat,
return true;
}
+static bool cat_check_cbm(const struct feat_node *feat, unsigned long cbm)
+{
+ unsigned int first_bit, zero_bit;
+ unsigned int cbm_len = feat->cat.cbm_len;
+
+ /*
+ * Set bits should only in the range of [0, cbm_len].
+ * And, at least one bit need to be set.
+ */
+ if ( cbm & (~0ul << cbm_len) || cbm == 0 )
+ return false;
+
+ first_bit = find_first_bit(&cbm, cbm_len);
+ zero_bit = find_next_zero_bit(&cbm, cbm_len, first_bit);
+
+ /* Set bits should be contiguous. */
+ if ( zero_bit < cbm_len &&
+ find_next_bit(&cbm, cbm_len, zero_bit) < cbm_len )
+ return false;
+
+ return true;
+}
+
/* L3 CAT props */
-static void l3_cat_write_msr(unsigned int cos, uint32_t val,
- enum psr_type type)
+static uint32_t l3_cat_write_msr(unsigned int cos, uint32_t val,
+ enum psr_type type)
{
wrmsrl(MSR_IA32_PSR_L3_MASK(cos), val);
+
+ return val;
}
static const struct feat_props l3_cat_props = {
@@ -439,6 +444,7 @@ static const struct feat_props l3_cat_props = {
.alt_type = PSR_TYPE_UNKNOWN,
.get_feat_info = cat_get_feat_info,
.write_msr = l3_cat_write_msr,
+ .check_val = cat_check_cbm,
};
/* L3 CDP props */
@@ -453,13 +459,15 @@ static bool l3_cdp_get_feat_info(const struct feat_node *feat,
return true;
}
-static void l3_cdp_write_msr(unsigned int cos, uint32_t val,
- enum psr_type type)
+static uint32_t l3_cdp_write_msr(unsigned int cos, uint32_t val,
+ enum psr_type type)
{
wrmsrl(((type == PSR_TYPE_L3_DATA) ?
MSR_IA32_PSR_L3_MASK_DATA(cos) :
MSR_IA32_PSR_L3_MASK_CODE(cos)),
val);
+
+ return val;
}
static const struct feat_props l3_cdp_props = {
@@ -469,13 +477,16 @@ static const struct feat_props l3_cdp_props = {
.alt_type = PSR_TYPE_L3_CBM,
.get_feat_info = l3_cdp_get_feat_info,
.write_msr = l3_cdp_write_msr,
+ .check_val = cat_check_cbm,
};
/* L2 CAT props */
-static void l2_cat_write_msr(unsigned int cos, uint32_t val,
- enum psr_type type)
+static uint32_t l2_cat_write_msr(unsigned int cos, uint32_t val,
+ enum psr_type type)
{
wrmsrl(MSR_IA32_PSR_L2_MASK(cos), val);
+
+ return val;
}
static const struct feat_props l2_cat_props = {
@@ -484,6 +495,7 @@ static const struct feat_props l2_cat_props = {
.alt_type = PSR_TYPE_UNKNOWN,
.get_feat_info = cat_get_feat_info,
.write_msr = l2_cat_write_msr,
+ .check_val = cat_check_cbm,
};
/* MBA props */
@@ -502,9 +514,23 @@ static bool mba_get_feat_info(const struct feat_node *feat,
return true;
}
-static void mba_write_msr(unsigned int cos, uint32_t val,
- enum psr_type type)
+static uint32_t mba_write_msr(unsigned int cos, uint32_t val,
+ enum psr_type type)
+{
+ wrmsrl(MSR_IA32_PSR_MBA_MASK(cos), val);
+
+ /* Read actual value set by hardware. */
+ rdmsrl(MSR_IA32_PSR_MBA_MASK(cos), val);
+
+ return val;
+}
+
+static bool mba_check_thrtl(const struct feat_node *feat, unsigned long thrtl)
{
+ if ( thrtl > feat->mba.thrtl_max )
+ return false;
+
+ return true;
}
static const struct feat_props mba_props = {
@@ -513,6 +539,7 @@ static const struct feat_props mba_props = {
.alt_type = PSR_TYPE_UNKNOWN,
.get_feat_info = mba_get_feat_info,
.write_msr = mba_write_msr,
+ .check_val = mba_check_thrtl,
};
static bool __init parse_psr_bool(const char *s, const char *delim,
@@ -978,7 +1005,7 @@ static int insert_val_into_array(uint32_t val[],
if ( array_len < props->cos_num )
return -ENOSPC;
- if ( !psr_check_cbm(feat->cat.cbm_len, new_val) )
+ if ( !props->check_val(feat, new_val) )
return -EINVAL;
/*
@@ -1210,25 +1237,39 @@ static unsigned int get_socket_cpu(unsigned int socket)
struct cos_write_info
{
unsigned int cos;
- struct feat_node *feature;
+ struct feat_node **features;
const uint32_t *val;
- const struct feat_props *props;
+ unsigned int array_len;
+ const struct feat_props **props;
};
static void do_write_psr_msrs(void *data)
{
const struct cos_write_info *info = data;
- struct feat_node *feat = info->feature;
- const struct feat_props *props = info->props;
- unsigned int i, cos = info->cos, cos_num = props->cos_num;
+ unsigned int i, index = 0, cos = info->cos;
+ const uint32_t *val_array = info->val;
- for ( i = 0; i < cos_num; i++ )
+ for ( i = 0; i < ARRAY_SIZE(feat_props); i++ )
{
- if ( feat->cos_reg_val[cos * cos_num + i] != info->val[i] )
+ struct feat_node *feat = info->features[i];
+ const struct feat_props *props = info->props[i];
+ unsigned int cos_num, j;
+
+ if ( !feat || !props )
+ continue;
+
+ cos_num = props->cos_num;
+ if ( info->array_len < index + cos_num )
+ return;
+
+ for ( j = 0; j < cos_num; j++ )
{
- feat->cos_reg_val[cos * cos_num + i] = info->val[i];
- props->write_msr(cos, info->val[i], props->type[i]);
+ if ( feat->cos_reg_val[cos * cos_num + j] != val_array[index + j] )
+ feat->cos_reg_val[cos * cos_num + j] =
+ props->write_msr(cos, val_array[index + j], props->type[j]);
}
+
+ index += cos_num;
}
}
@@ -1236,30 +1277,19 @@ static int write_psr_msrs(unsigned int socket, unsigned int cos,
const uint32_t val[], unsigned int array_len,
enum psr_feat_type feat_type)
{
- int ret;
struct psr_socket_info *info = get_socket_info(socket);
struct cos_write_info data =
{
.cos = cos,
- .feature = info->features[feat_type],
- .props = feat_props[feat_type],
+ .features = info->features,
+ .val = val,
+ .array_len = array_len,
+ .props = feat_props,
};
if ( cos > info->features[feat_type]->cos_max )
return -EINVAL;
- /* Skip to the feature's value head. */
- ret = skip_prior_features(&array_len, feat_type);
- if ( ret < 0 )
- return ret;
-
- val += ret;
-
- if ( array_len < feat_props[feat_type]->cos_num )
- return -ENOSPC;
-
- data.val = val;
-
if ( socket == cpu_to_socket(smp_processor_id()) )
do_write_psr_msrs(&data);
else
diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
index ec6d2de..714b3e8 100644
--- a/xen/include/public/domctl.h
+++ b/xen/include/public/domctl.h
@@ -1156,6 +1156,7 @@ struct xen_domctl_psr_alloc {
#define XEN_DOMCTL_PSR_GET_L3_DATA 5
#define XEN_DOMCTL_PSR_SET_L2_CBM 6
#define XEN_DOMCTL_PSR_GET_L2_CBM 7
+#define XEN_DOMCTL_PSR_SET_MBA_THRTL 8
#define XEN_DOMCTL_PSR_GET_MBA_THRTL 9
uint32_t cmd; /* IN: XEN_DOMCTL_PSR_* */
uint32_t target; /* IN */
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v4 08/15] tools: create general interfaces to support psr allocation features
2017-09-23 9:48 [PATCH v4 00/15] Enable Memory Bandwidth Allocation in Xen Yi Sun
` (6 preceding siblings ...)
2017-09-23 9:48 ` [PATCH v4 07/15] x86: implement set value flow " Yi Sun
@ 2017-09-23 9:48 ` Yi Sun
2017-09-26 10:11 ` Roger Pau Monné
2017-09-23 9:48 ` [PATCH v4 09/15] tools: implement the new libxc get hw info interface Yi Sun
` (8 subsequent siblings)
16 siblings, 1 reply; 64+ messages in thread
From: Yi Sun @ 2017-09-23 9:48 UTC (permalink / raw)
To: xen-devel; +Cc: Wei Liu, Chao Peng, Yi Sun, Ian Jackson, Roger Pau Monné
This patch creates general interfaces in libxl to support all psr
allocation features.
Add 'LIBXL_HAVE_PSR_GENERIC' to indicate interface change.
Please note, the functionality cannot work until later patches
are applied.
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
---
CC: Wei Liu <wei.liu2@citrix.com>
CC: Ian Jackson <ian.jackson@eu.citrix.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Chao Peng <chao.p.peng@linux.intel.com>
v4:
- add description for LIBXL_HAVE_PSR_GENERIC to mention newly added
public functions.
(suggested by Roger Pau Monné)
v3:
- change 'LIBXL_HAVE_PSR_MBA' to 'LIBXL_HAVE_PSR_GENERIC'.
(suggested by Roger Pau Monné)
- 'typedef enum libxl_psr_cbm_type libxl_psr_type;' in libxl.h.
(suggested by Roger Pau Monné and Wei Liu)
- change 'libxl_psr_cbm_type' to 'libxl_psr_type' in newly defined
interfaces.
(suggested by Roger Pau Monné)
v2:
- remove '_INFO' in 'libxl_psr_feat_type' and make corresponding
changes in 'libxl_psr_hw_info'.
(suggested by Chao Peng)
---
tools/libxl/libxl.h | 37 +++++++++++++++++++++++++++++++++++++
tools/libxl/libxl_psr.c | 25 +++++++++++++++++++++++++
tools/libxl/libxl_types.idl | 22 ++++++++++++++++++++++
3 files changed, 84 insertions(+)
diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h
index 91408b4..569c331 100644
--- a/tools/libxl/libxl.h
+++ b/tools/libxl/libxl.h
@@ -936,6 +936,17 @@ void libxl_mac_copy(libxl_ctx *ctx, libxl_mac *dst, const libxl_mac *src);
#define LIBXL_HAVE_PSR_L2_CAT 1
/*
+ * LIBXL_HAVE_PSR_GENERIC
+ *
+ * If this is defined, the Memory Bandwidth Allocation feature is supported.
+ * The following public functions are available:
+ * libxl_psr_{set/get}_val
+ * libxl_psr_get_hw_info
+ * libxl_psr_hw_info_list_free
+ */
+#define LIBXL_HAVE_PSR_GENERIC 1
+
+/*
* LIBXL_HAVE_MCA_CAPS
*
* If this is defined, setting MCA capabilities for HVM domain is supported.
@@ -2228,6 +2239,32 @@ int libxl_psr_cat_get_info(libxl_ctx *ctx, libxl_psr_cat_info **info,
int libxl_psr_cat_get_l3_info(libxl_ctx *ctx, libxl_psr_cat_info **info,
int *nr);
void libxl_psr_cat_info_list_free(libxl_psr_cat_info *list, int nr);
+
+typedef enum libxl_psr_cbm_type libxl_psr_type;
+
+/*
+ * Function to set a domain's value. It operates on a single or multiple
+ * target(s) defined in 'target_map'. 'target_map' specifies all the sockets
+ * to be operated on.
+ */
+int libxl_psr_set_val(libxl_ctx *ctx, uint32_t domid,
+ libxl_psr_type type, libxl_bitmap *target_map,
+ uint64_t val);
+/*
+ * Function to get a domain's cbm. It operates on a single 'target'.
+ * 'target' specifies which socket to be operated on.
+ */
+int libxl_psr_get_val(libxl_ctx *ctx, uint32_t domid,
+ libxl_psr_type type, unsigned int target,
+ uint64_t *val);
+/*
+ * On success, the function returns an array of elements in 'info',
+ * and the length in 'nr'.
+ */
+int libxl_psr_get_hw_info(libxl_ctx *ctx, libxl_psr_hw_info **info,
+ unsigned int *nr, libxl_psr_feat_type type,
+ unsigned int lvl);
+void libxl_psr_hw_info_list_free(libxl_psr_hw_info *list, unsigned int nr);
#endif
/* misc */
diff --git a/tools/libxl/libxl_psr.c b/tools/libxl/libxl_psr.c
index 197505a..4a6978e 100644
--- a/tools/libxl/libxl_psr.c
+++ b/tools/libxl/libxl_psr.c
@@ -428,6 +428,31 @@ void libxl_psr_cat_info_list_free(libxl_psr_cat_info *list, int nr)
free(list);
}
+int libxl_psr_set_val(libxl_ctx *ctx, uint32_t domid,
+ libxl_psr_type type, libxl_bitmap *target_map,
+ uint64_t val)
+{
+ return ERROR_FAIL;
+}
+
+int libxl_psr_get_val(libxl_ctx *ctx, uint32_t domid,
+ libxl_psr_type type, unsigned int target,
+ uint64_t *val)
+{
+ return ERROR_FAIL;
+}
+
+int libxl_psr_get_hw_info(libxl_ctx *ctx, libxl_psr_hw_info **info,
+ unsigned int *nr, libxl_psr_feat_type type,
+ unsigned int lvl)
+{
+ return ERROR_FAIL;
+}
+
+void libxl_psr_hw_info_list_free(libxl_psr_hw_info *list, unsigned int nr)
+{
+}
+
/*
* Local variables:
* mode: C
diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl
index 173d70a..cfe8367 100644
--- a/tools/libxl/libxl_types.idl
+++ b/tools/libxl/libxl_types.idl
@@ -977,6 +977,7 @@ libxl_psr_cbm_type = Enumeration("psr_cbm_type", [
(2, "L3_CBM_CODE"),
(3, "L3_CBM_DATA"),
(4, "L2_CBM"),
+ (5, "MBA_THRTL"),
])
libxl_psr_cat_info = Struct("psr_cat_info", [
@@ -985,3 +986,24 @@ libxl_psr_cat_info = Struct("psr_cat_info", [
("cbm_len", uint32),
("cdp_enabled", bool),
])
+
+libxl_psr_feat_type = Enumeration("psr_feat_type", [
+ (1, "CAT"),
+ (2, "MBA"),
+ ])
+
+libxl_psr_hw_info = Struct("psr_hw_info", [
+ ("id", uint32),
+ ("u", KeyedUnion(None, libxl_psr_feat_type, "type",
+ [("cat", Struct(None, [
+ ("cos_max", uint32),
+ ("cbm_len", uint32),
+ ("cdp_enabled", bool),
+ ])),
+ ("mba", Struct(None, [
+ ("cos_max", uint32),
+ ("thrtl_max", uint32),
+ ("linear", bool),
+ ])),
+ ]))
+ ], dir=DIR_OUT)
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v4 09/15] tools: implement the new libxc get hw info interface
2017-09-23 9:48 [PATCH v4 00/15] Enable Memory Bandwidth Allocation in Xen Yi Sun
` (7 preceding siblings ...)
2017-09-23 9:48 ` [PATCH v4 08/15] tools: create general interfaces to support psr allocation features Yi Sun
@ 2017-09-23 9:48 ` Yi Sun
2017-09-26 10:22 ` Roger Pau Monné
2017-09-23 9:48 ` [PATCH v4 10/15] tools: implement the new libxl " Yi Sun
` (7 subsequent siblings)
16 siblings, 1 reply; 64+ messages in thread
From: Yi Sun @ 2017-09-23 9:48 UTC (permalink / raw)
To: xen-devel; +Cc: Wei Liu, Chao Peng, Yi Sun, Ian Jackson, Roger Pau Monné
This patch implements a new libxc get hw info interface and corresponding
data structures. It also changes libxl_psr.c to call this new interface.
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
---
CC: Wei Liu <wei.liu2@citrix.com>
CC: Ian Jackson <ian.jackson@eu.citrix.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Chao Peng <chao.p.peng@linux.intel.com>
v4:
- remove 'ALLOC_' from macro names.
(suggested by Roger Pau Monné)
- remove 'XC_PSR_FEAT_UNKNOWN' which is not necessary.
(suggested by Roger Pau Monné)
- remove 'FEAT_' from enum item names.
(suggested by Roger Pau Monné)
- remove 'xc_' from struct name.
(suggested by Roger Pau Monné)
- adjust codes to reduce indentation.
(suggested by Roger Pau Monné)
- assert for not happened case.
(suggested by Roger Pau Monné)
- add LOGE to show errno.
(suggested by Roger Pau Monné)
v3:
- rename 'psr_cat/PSR_CAT' to 'psr_alloc/PSR_ALLOC' and remove 'op/OP'
from name.
(suggested by Roger Pau Monné)
- remove 'info' from 'xc_cat_info' and 'xc_mba_info'.
(suggested by Roger Pau Monné)
- set errno in 'xc_psr_get_hw_info'.
(suggested by Roger Pau Monné)
- remove 'inline'.
(suggested by Roger Pau Monné)
- remove 'psr' from 'libxl__psr_feat_type_to_libxc_psr_feat_type' to make
function name shorter.
(suggested by Roger Pau Monné)
- check 'xc_type' in 'libxl_psr_cat_get_info'.
(suggested by Roger Pau Monné)
v2:
- split this patch out from a big patch in v1.
(suggested by Wei Liu)
- change 'CAT_INFO' and 'MBA_INFO' to 'CAT' and 'MBA'.
(suggested by Chao Peng)
---
tools/libxc/include/xenctrl.h | 29 +++++++++++++++++++---
tools/libxc/xc_psr.c | 56 +++++++++++++++++++++++++++++--------------
tools/libxl/libxl_psr.c | 36 ++++++++++++++++++++++++++--
3 files changed, 98 insertions(+), 23 deletions(-)
diff --git a/tools/libxc/include/xenctrl.h b/tools/libxc/include/xenctrl.h
index 43151cb..58ba87c 100644
--- a/tools/libxc/include/xenctrl.h
+++ b/tools/libxc/include/xenctrl.h
@@ -2459,6 +2459,30 @@ enum xc_psr_cat_type {
};
typedef enum xc_psr_cat_type xc_psr_cat_type;
+enum xc_psr_feat_type {
+ XC_PSR_CAT_L3,
+ XC_PSR_CAT_L2,
+ XC_PSR_MBA,
+};
+typedef enum xc_psr_feat_type xc_psr_feat_type;
+
+struct xc_psr_hw_info {
+ union {
+ struct {
+ uint32_t cos_max;
+ uint32_t cbm_len;
+ bool cdp_enabled;
+ } cat;
+
+ struct {
+ uint32_t cos_max;
+ uint32_t thrtl_max;
+ bool linear;
+ } mba;
+ } u;
+};
+typedef struct xc_psr_hw_info xc_psr_hw_info;
+
int xc_psr_cmt_attach(xc_interface *xch, uint32_t domid);
int xc_psr_cmt_detach(xc_interface *xch, uint32_t domid);
int xc_psr_cmt_get_domain_rmid(xc_interface *xch, uint32_t domid,
@@ -2480,9 +2504,8 @@ int xc_psr_cat_set_domain_data(xc_interface *xch, uint32_t domid,
int xc_psr_cat_get_domain_data(xc_interface *xch, uint32_t domid,
xc_psr_cat_type type, uint32_t target,
uint64_t *data);
-int xc_psr_cat_get_info(xc_interface *xch, uint32_t socket, unsigned int lvl,
- uint32_t *cos_max, uint32_t *cbm_len,
- bool *cdp_enabled);
+int xc_psr_get_hw_info(xc_interface *xch, uint32_t socket,
+ xc_psr_feat_type type, xc_psr_hw_info *hw_info);
int xc_get_cpu_levelling_caps(xc_interface *xch, uint32_t *caps);
int xc_get_cpu_featureset(xc_interface *xch, uint32_t index,
diff --git a/tools/libxc/xc_psr.c b/tools/libxc/xc_psr.c
index 623e26b..f2b5ee6 100644
--- a/tools/libxc/xc_psr.c
+++ b/tools/libxc/xc_psr.c
@@ -323,37 +323,57 @@ int xc_psr_cat_get_domain_data(xc_interface *xch, uint32_t domid,
return rc;
}
-int xc_psr_cat_get_info(xc_interface *xch, uint32_t socket, unsigned int lvl,
- uint32_t *cos_max, uint32_t *cbm_len, bool *cdp_enabled)
+int xc_psr_get_hw_info(xc_interface *xch, uint32_t socket,
+ xc_psr_feat_type type, xc_psr_hw_info *hw_info)
{
int rc = -1;
DECLARE_SYSCTL;
+ if ( !hw_info )
+ {
+ errno = EINVAL;
+ return rc;
+ }
+
sysctl.cmd = XEN_SYSCTL_psr_alloc;
sysctl.u.psr_alloc.target = socket;
- switch ( lvl )
+ switch ( type )
{
- case 2:
+ case XC_PSR_CAT_L2:
sysctl.u.psr_alloc.cmd = XEN_SYSCTL_PSR_get_l2_info;
rc = xc_sysctl(xch, &sysctl);
- if ( !rc )
- {
- *cos_max = sysctl.u.psr_alloc.u.cat_info.cos_max;
- *cbm_len = sysctl.u.psr_alloc.u.cat_info.cbm_len;
- *cdp_enabled = false;
- }
+ if ( rc )
+ break;
+
+ hw_info->u.cat.cos_max = sysctl.u.psr_alloc.u.cat_info.cos_max;
+ hw_info->u.cat.cbm_len = sysctl.u.psr_alloc.u.cat_info.cbm_len;
+ hw_info->u.cat.cdp_enabled = false;
+
break;
- case 3:
+ case XC_PSR_CAT_L3:
sysctl.u.psr_alloc.cmd = XEN_SYSCTL_PSR_get_l3_info;
rc = xc_sysctl(xch, &sysctl);
- if ( !rc )
- {
- *cos_max = sysctl.u.psr_alloc.u.cat_info.cos_max;
- *cbm_len = sysctl.u.psr_alloc.u.cat_info.cbm_len;
- *cdp_enabled = sysctl.u.psr_alloc.u.cat_info.flags &
- XEN_SYSCTL_PSR_L3_CDP;
- }
+ if ( rc )
+ break;
+
+ hw_info->u.cat.cos_max = sysctl.u.psr_alloc.u.cat_info.cos_max;
+ hw_info->u.cat.cbm_len = sysctl.u.psr_alloc.u.cat_info.cbm_len;
+ hw_info->u.cat.cdp_enabled = sysctl.u.psr_alloc.u.cat_info.flags &
+ XEN_SYSCTL_PSR_L3_CDP;
+
+ break;
+ case XC_PSR_MBA:
+ sysctl.u.psr_alloc.cmd = XEN_SYSCTL_PSR_get_mba_info;
+ rc = xc_sysctl(xch, &sysctl);
+ if ( rc )
+ break;
+
+ hw_info->u.mba.cos_max = sysctl.u.psr_alloc.u.mba_info.cos_max;
+ hw_info->u.mba.thrtl_max = sysctl.u.psr_alloc.u.mba_info.thrtl_max;
+ hw_info->u.mba.linear = sysctl.u.psr_alloc.u.mba_info.flags &
+ XEN_SYSCTL_PSR_MBA_LINEAR;
+
break;
default:
errno = EOPNOTSUPP;
diff --git a/tools/libxl/libxl_psr.c b/tools/libxl/libxl_psr.c
index 4a6978e..7349490 100644
--- a/tools/libxl/libxl_psr.c
+++ b/tools/libxl/libxl_psr.c
@@ -361,6 +361,29 @@ int libxl_psr_cat_get_cbm(libxl_ctx *ctx, uint32_t domid,
return rc;
}
+static xc_psr_feat_type libxl__feat_type_to_libxc_feat_type(
+ libxl_psr_feat_type type, unsigned int lvl)
+{
+ xc_psr_feat_type xc_type;
+
+ switch (type) {
+ case LIBXL_PSR_FEAT_TYPE_CAT:
+ if (lvl == 3)
+ xc_type = XC_PSR_CAT_L3;
+ if (lvl == 2)
+ xc_type = XC_PSR_CAT_L2;
+ break;
+ case LIBXL_PSR_FEAT_TYPE_MBA:
+ xc_type = XC_PSR_MBA;
+ break;
+ default:
+ /* Could not happen */
+ assert(0);
+ }
+
+ return xc_type;
+}
+
int libxl_psr_cat_get_info(libxl_ctx *ctx, libxl_psr_cat_info **info,
unsigned int *nr, unsigned int lvl)
{
@@ -369,6 +392,8 @@ int libxl_psr_cat_get_info(libxl_ctx *ctx, libxl_psr_cat_info **info,
int i = 0, socketid, nr_sockets;
libxl_bitmap socketmap;
libxl_psr_cat_info *ptr;
+ xc_psr_hw_info hw_info;
+ xc_psr_feat_type xc_type;
libxl_bitmap_init(&socketmap);
@@ -385,16 +410,23 @@ int libxl_psr_cat_get_info(libxl_ctx *ctx, libxl_psr_cat_info **info,
goto out;
}
+ xc_type = libxl__feat_type_to_libxc_feat_type(LIBXL_PSR_FEAT_TYPE_CAT, lvl);
+
ptr = libxl__malloc(NOGC, nr_sockets * sizeof(libxl_psr_cat_info));
libxl_for_each_set_bit(socketid, socketmap) {
ptr[i].id = socketid;
- if (xc_psr_cat_get_info(ctx->xch, socketid, lvl, &ptr[i].cos_max,
- &ptr[i].cbm_len, &ptr[i].cdp_enabled)) {
+ if (xc_psr_get_hw_info(ctx->xch, socketid, xc_type, &hw_info)) {
+ LOGE(ERROR, "failed to get hw info");
rc = ERROR_FAIL;
free(ptr);
goto out;
}
+
+ ptr[i].cos_max = hw_info.u.cat.cos_max;
+ ptr[i].cbm_len = hw_info.u.cat.cbm_len;
+ ptr[i].cdp_enabled = hw_info.u.cat.cdp_enabled;
+
i++;
}
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v4 10/15] tools: implement the new libxl get hw info interface
2017-09-23 9:48 [PATCH v4 00/15] Enable Memory Bandwidth Allocation in Xen Yi Sun
` (8 preceding siblings ...)
2017-09-23 9:48 ` [PATCH v4 09/15] tools: implement the new libxc get hw info interface Yi Sun
@ 2017-09-23 9:48 ` Yi Sun
2017-09-26 10:54 ` Roger Pau Monné
2017-09-23 9:48 ` [PATCH v4 11/15] tools: implement the new xl " Yi Sun
` (6 subsequent siblings)
16 siblings, 1 reply; 64+ messages in thread
From: Yi Sun @ 2017-09-23 9:48 UTC (permalink / raw)
To: xen-devel; +Cc: Wei Liu, Chao Peng, Yi Sun, Ian Jackson, Roger Pau Monné
This patch implements the new libxl get hw info interface,
'libxl_psr_get_hw_info', which is suitable to all psr allocation
features. It also implements corresponding list free function,
'libxl_psr_hw_info_list_free' and makes 'libxl_psr_cat_get_info' call
'libxl_psr_get_hw_info' to avoid redundant code in libxl_psr.c.
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
---
CC: Wei Liu <wei.liu2@citrix.com>
CC: Ian Jackson <ian.jackson@eu.citrix.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Chao Peng <chao.p.peng@linux.intel.com>
v4:
- remove 'xc_' from struct name.
(suggested by Roger Pau Monné)
- fix words in commit message.
(suggested by Roger Pau Monné)
- change type of 'libxl__hw_info_to_libxl_cat_info' to void and use
assert to check invalid type. Then, remove check for
'libxl__hw_info_to_libxl_cat_info'.
(suggested by Roger Pau Monné)
- change type of 'libxl__xc_hw_info_to_libxl_hw_info' to void and use
assert to check invalid type. Then, remove check for
'libxl__xc_hw_info_to_libxl_hw_info'.
(suggested by Roger Pau Monné)
v3:
- remove casting.
(suggested by Roger Pau Monné)
- remove inline.
(suggested by Roger Pau Monné)
- change 'libxc__psr_hw_info_to_libxl_psr_hw_info' to
'libxl__xc_hw_info_to_libxl_hw_info'.
(suggested by Roger Pau Monné)
- remove '_hw' from parameter names.
(suggested by Roger Pau Monné)
- change some 'LOGE' to 'LOG'.
(suggested by Roger Pau Monné)
- check returned 'xc_type' and remove redundant 'lvl' check.
(suggested by Roger Pau Monné)
v2:
- split this patch out from a big patch in v1.
(suggested by Wei Liu)
- change 'CAT_INFO'/'MBA_INFO' to 'CAT' and 'MBA. Also the libxl structure
name 'cat_info'/'mba_info' is changed to 'cat'/'mba'.
(suggested by Chao Peng)
- call 'libxl_psr_hw_info_list_free' in 'libxl_psr_cat_get_info' to free
allocated resources.
(suggested by Chao Peng)
---
tools/libxl/libxl_psr.c | 131 ++++++++++++++++++++++++++++++++++--------------
1 file changed, 93 insertions(+), 38 deletions(-)
diff --git a/tools/libxl/libxl_psr.c b/tools/libxl/libxl_psr.c
index 7349490..570d21a 100644
--- a/tools/libxl/libxl_psr.c
+++ b/tools/libxl/libxl_psr.c
@@ -384,56 +384,41 @@ static xc_psr_feat_type libxl__feat_type_to_libxc_feat_type(
return xc_type;
}
+static void libxl__hw_info_to_libxl_cat_info(
+ libxl_psr_feat_type type, libxl_psr_hw_info *hw_info,
+ libxl_psr_cat_info *cat_info)
+{
+ assert(type == LIBXL_PSR_FEAT_TYPE_CAT);
+
+ cat_info->id = hw_info->id;
+ cat_info->cos_max = hw_info->u.cat.cos_max;
+ cat_info->cbm_len = hw_info->u.cat.cbm_len;
+ cat_info->cdp_enabled = hw_info->u.cat.cdp_enabled;
+}
+
int libxl_psr_cat_get_info(libxl_ctx *ctx, libxl_psr_cat_info **info,
unsigned int *nr, unsigned int lvl)
{
GC_INIT(ctx);
int rc;
- int i = 0, socketid, nr_sockets;
- libxl_bitmap socketmap;
+ unsigned int i;
+ libxl_psr_hw_info *hw_info;
libxl_psr_cat_info *ptr;
- xc_psr_hw_info hw_info;
- xc_psr_feat_type xc_type;
-
- libxl_bitmap_init(&socketmap);
-
- rc = libxl__count_physical_sockets(gc, &nr_sockets);
- if (rc) {
- LOGE(ERROR, "failed to get system socket count");
- goto out;
- }
- libxl_socket_bitmap_alloc(ctx, &socketmap, nr_sockets);
- rc = libxl_get_online_socketmap(ctx, &socketmap);
- if (rc < 0) {
- LOGE(ERROR, "failed to get available sockets");
+ rc = libxl_psr_get_hw_info(ctx, &hw_info, nr, LIBXL_PSR_FEAT_TYPE_CAT, lvl);
+ if (rc)
goto out;
- }
-
- xc_type = libxl__feat_type_to_libxc_feat_type(LIBXL_PSR_FEAT_TYPE_CAT, lvl);
-
- ptr = libxl__malloc(NOGC, nr_sockets * sizeof(libxl_psr_cat_info));
-
- libxl_for_each_set_bit(socketid, socketmap) {
- ptr[i].id = socketid;
- if (xc_psr_get_hw_info(ctx->xch, socketid, xc_type, &hw_info)) {
- LOGE(ERROR, "failed to get hw info");
- rc = ERROR_FAIL;
- free(ptr);
- goto out;
- }
- ptr[i].cos_max = hw_info.u.cat.cos_max;
- ptr[i].cbm_len = hw_info.u.cat.cbm_len;
- ptr[i].cdp_enabled = hw_info.u.cat.cdp_enabled;
+ ptr = libxl__malloc(NOGC, *nr * sizeof(libxl_psr_cat_info));
- i++;
- }
+ for (i = 0; i < *nr; i++)
+ libxl__hw_info_to_libxl_cat_info(LIBXL_PSR_FEAT_TYPE_CAT,
+ &hw_info[i],
+ &ptr[i]);
*info = ptr;
- *nr = i;
+ libxl_psr_hw_info_list_free(hw_info, *nr);
out:
- libxl_bitmap_dispose(&socketmap);
GC_FREE;
return rc;
}
@@ -474,15 +459,85 @@ int libxl_psr_get_val(libxl_ctx *ctx, uint32_t domid,
return ERROR_FAIL;
}
+static void libxl__xc_hw_info_to_libxl_hw_info(
+ libxl_psr_feat_type type, xc_psr_hw_info *xc_info,
+ libxl_psr_hw_info *xl_info)
+{
+ switch (type) {
+ case LIBXL_PSR_FEAT_TYPE_CAT:
+ xl_info->u.cat.cos_max = xc_info->u.cat.cos_max;
+ xl_info->u.cat.cbm_len = xc_info->u.cat.cbm_len;
+ xl_info->u.cat.cdp_enabled = xc_info->u.cat.cdp_enabled;
+ break;
+ case LIBXL_PSR_FEAT_TYPE_MBA:
+ xl_info->u.mba.cos_max = xc_info->u.mba.cos_max;
+ xl_info->u.mba.thrtl_max = xc_info->u.mba.thrtl_max;
+ xl_info->u.mba.linear = xc_info->u.mba.linear;
+ break;
+ default:
+ assert(0);
+ }
+}
+
int libxl_psr_get_hw_info(libxl_ctx *ctx, libxl_psr_hw_info **info,
unsigned int *nr, libxl_psr_feat_type type,
unsigned int lvl)
{
- return ERROR_FAIL;
+ GC_INIT(ctx);
+ int rc, nr_sockets;
+ unsigned int i = 0, socketid;
+ libxl_bitmap socketmap;
+ libxl_psr_hw_info *ptr;
+ xc_psr_feat_type xc_type;
+ xc_psr_hw_info hw_info;
+
+ libxl_bitmap_init(&socketmap);
+
+ xc_type = libxl__feat_type_to_libxc_feat_type(type, lvl);
+
+ rc = libxl__count_physical_sockets(gc, &nr_sockets);
+ if (rc) {
+ LOG(ERROR, "failed to get system socket count");
+ goto out;
+ }
+
+ libxl_socket_bitmap_alloc(ctx, &socketmap, nr_sockets);
+ rc = libxl_get_online_socketmap(ctx, &socketmap);
+ if (rc < 0) {
+ LOGE(ERROR, "failed to get available sockets");
+ goto out;
+ }
+
+ ptr = libxl__malloc(NOGC, nr_sockets * sizeof(libxl_psr_hw_info));
+
+ libxl_for_each_set_bit(socketid, socketmap) {
+ ptr[i].id = socketid;
+ if (xc_psr_get_hw_info(ctx->xch, socketid, xc_type, &hw_info)) {
+ rc = ERROR_FAIL;
+ free(ptr);
+ goto out;
+ }
+
+ libxl__xc_hw_info_to_libxl_hw_info(type, &hw_info, &ptr[i]);
+
+ i++;
+ }
+
+ *info = ptr;
+ *nr = i;
+out:
+ libxl_bitmap_dispose(&socketmap);
+ GC_FREE;
+ return rc;
}
void libxl_psr_hw_info_list_free(libxl_psr_hw_info *list, unsigned int nr)
{
+ unsigned int i;
+
+ for (i = 0; i < nr; i++)
+ libxl_psr_hw_info_dispose(&list[i]);
+ free(list);
}
/*
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v4 11/15] tools: implement the new xl get hw info interface
2017-09-23 9:48 [PATCH v4 00/15] Enable Memory Bandwidth Allocation in Xen Yi Sun
` (9 preceding siblings ...)
2017-09-23 9:48 ` [PATCH v4 10/15] tools: implement the new libxl " Yi Sun
@ 2017-09-23 9:48 ` Yi Sun
2017-09-26 11:19 ` Roger Pau Monné
2017-09-23 9:48 ` [PATCH v4 12/15] tools: rename 'xc_psr_cat_type' to 'xc_psr_type' Yi Sun
` (5 subsequent siblings)
16 siblings, 1 reply; 64+ messages in thread
From: Yi Sun @ 2017-09-23 9:48 UTC (permalink / raw)
To: xen-devel; +Cc: Wei Liu, Chao Peng, Yi Sun, Ian Jackson, Roger Pau Monné
This patch implements a new xl get HW info interface. A new argument
is added for psr-hwinfo command to get and show MBA HW info.
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
---
CC: Wei Liu <wei.liu2@citrix.com>
CC: Ian Jackson <ian.jackson@eu.citrix.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Chao Peng <chao.p.peng@linux.intel.com>
v3:
- change the format string of printf in 'psr_mba_hwinfo'.
(suggested by Roger Pau Monné)
- add 'const' for 'opts[]' in 'main_psr_hwinfo'.
(suggested by Roger Pau Monné)
v2:
- split out this patch from a big patch in v1.
(suggested by Wei Liu)
- change 'MBA_INFO' to 'MBA'. Also, change 'mba_info' to 'mba'.
(suggested by Chao Peng)
---
tools/xl/xl_cmdtable.c | 1 +
tools/xl/xl_psr.c | 40 +++++++++++++++++++++++++++++++++++++---
2 files changed, 38 insertions(+), 3 deletions(-)
diff --git a/tools/xl/xl_cmdtable.c b/tools/xl/xl_cmdtable.c
index 5cfc7e8..a1679e9 100644
--- a/tools/xl/xl_cmdtable.c
+++ b/tools/xl/xl_cmdtable.c
@@ -529,6 +529,7 @@ struct cmd_spec cmd_table[] = {
"[options]",
"-m, --cmt Show Cache Monitoring Technology (CMT) hardware info\n"
"-a, --cat Show Cache Allocation Technology (CAT) hardware info\n"
+ "-b, --mba Show Memory Bandwidth Allocation (MBA) hardware info\n"
},
{ "psr-cmt-attach",
&main_psr_cmt_attach, 0, 1,
diff --git a/tools/xl/xl_psr.c b/tools/xl/xl_psr.c
index ef00048..40269b4 100644
--- a/tools/xl/xl_psr.c
+++ b/tools/xl/xl_psr.c
@@ -475,6 +475,32 @@ static int psr_l2_cat_hwinfo(void)
return rc;
}
+static int psr_mba_hwinfo(void)
+{
+ int rc;
+ unsigned int i, nr;
+ libxl_psr_hw_info *info;
+
+ rc = libxl_psr_get_hw_info(ctx, &info, &nr,
+ LIBXL_PSR_FEAT_TYPE_MBA, 0);
+ if (rc)
+ return rc;
+
+ printf("Memory Bandwidth Allocation (MBA):\n");
+
+ for (i = 0; i < nr; i++) {
+ printf("Socket ID : %u\n", info[i].id);
+ printf("Linear Mode : %s\n",
+ info[i].u.mba.linear ? "Enabled" : "Disabled");
+ printf("Maximum COS : %u\n", info[i].u.mba.cos_max);
+ printf("Maximum Throttling Value: %u\n", info[i].u.mba.thrtl_max);
+ printf("Default Throttling Value: %u\n", 0);
+ }
+
+ libxl_psr_hw_info_list_free(info, nr);
+ return rc;
+}
+
int main_psr_cat_cbm_set(int argc, char **argv)
{
uint32_t domid;
@@ -593,20 +619,24 @@ int main_psr_cat_show(int argc, char **argv)
int main_psr_hwinfo(int argc, char **argv)
{
int opt, ret = 0;
- bool all = true, cmt = false, cat = false;
- static struct option opts[] = {
+ bool all = true, cmt = false, cat = false, mba = false;
+ static const struct option opts[] = {
{"cmt", 0, 0, 'm'},
{"cat", 0, 0, 'a'},
+ {"mba", 0, 0, 'b'},
COMMON_LONG_OPTS
};
- SWITCH_FOREACH_OPT(opt, "ma", opts, "psr-hwinfo", 0) {
+ SWITCH_FOREACH_OPT(opt, "mab", opts, "psr-hwinfo", 0) {
case 'm':
all = false; cmt = true;
break;
case 'a':
all = false; cat = true;
break;
+ case 'b':
+ all = false; mba = true;
+ break;
}
if (!ret && (all || cmt))
@@ -619,6 +649,10 @@ int main_psr_hwinfo(int argc, char **argv)
if (all || cat)
ret = psr_l2_cat_hwinfo();
+ /* MBA is independent of CMT and CAT */
+ if (all || mba)
+ ret = psr_mba_hwinfo();
+
return ret;
}
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v4 12/15] tools: rename 'xc_psr_cat_type' to 'xc_psr_type'
2017-09-23 9:48 [PATCH v4 00/15] Enable Memory Bandwidth Allocation in Xen Yi Sun
` (10 preceding siblings ...)
2017-09-23 9:48 ` [PATCH v4 11/15] tools: implement the new xl " Yi Sun
@ 2017-09-23 9:48 ` Yi Sun
2017-09-26 11:21 ` Roger Pau Monné
2017-09-23 9:48 ` [PATCH v4 13/15] tools: implement new generic get value interface and MBA get value command Yi Sun
` (4 subsequent siblings)
16 siblings, 1 reply; 64+ messages in thread
From: Yi Sun @ 2017-09-23 9:48 UTC (permalink / raw)
To: xen-devel; +Cc: Wei Liu, Chao Peng, Yi Sun, Ian Jackson, Roger Pau Monné
This patch renames 'xc_psr_cat_type' to 'xc_psr_type' so that
the structure name is common for all allocation features.
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Chao Peng <chao.p.peng@linux.intel.com>
---
CC: Wei Liu <wei.liu2@citrix.com>
CC: Ian Jackson <ian.jackson@eu.citrix.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Chao Peng <chao.p.peng@linux.intel.com>
v4:
- move assignment of xc_type to its declaration place.
(suggested by Roger Pau Monné)
v3:
- change 'xc_psr_val_type' to 'xc_psr_type'.
(suggested by Roger Pau Monné)
---
tools/libxc/include/xenctrl.h | 8 ++++----
tools/libxc/xc_psr.c | 4 ++--
tools/libxl/libxl_psr.c | 11 +++++------
3 files changed, 11 insertions(+), 12 deletions(-)
diff --git a/tools/libxc/include/xenctrl.h b/tools/libxc/include/xenctrl.h
index 58ba87c..1cc4d7e 100644
--- a/tools/libxc/include/xenctrl.h
+++ b/tools/libxc/include/xenctrl.h
@@ -2451,13 +2451,13 @@ enum xc_psr_cmt_type {
};
typedef enum xc_psr_cmt_type xc_psr_cmt_type;
-enum xc_psr_cat_type {
+enum xc_psr_type {
XC_PSR_CAT_L3_CBM = 1,
XC_PSR_CAT_L3_CBM_CODE = 2,
XC_PSR_CAT_L3_CBM_DATA = 3,
XC_PSR_CAT_L2_CBM = 4,
};
-typedef enum xc_psr_cat_type xc_psr_cat_type;
+typedef enum xc_psr_type xc_psr_type;
enum xc_psr_feat_type {
XC_PSR_CAT_L3,
@@ -2499,10 +2499,10 @@ int xc_psr_cmt_get_data(xc_interface *xch, uint32_t rmid, uint32_t cpu,
int xc_psr_cmt_enabled(xc_interface *xch);
int xc_psr_cat_set_domain_data(xc_interface *xch, uint32_t domid,
- xc_psr_cat_type type, uint32_t target,
+ xc_psr_type type, uint32_t target,
uint64_t data);
int xc_psr_cat_get_domain_data(xc_interface *xch, uint32_t domid,
- xc_psr_cat_type type, uint32_t target,
+ xc_psr_type type, uint32_t target,
uint64_t *data);
int xc_psr_get_hw_info(xc_interface *xch, uint32_t socket,
xc_psr_feat_type type, xc_psr_hw_info *hw_info);
diff --git a/tools/libxc/xc_psr.c b/tools/libxc/xc_psr.c
index f2b5ee6..385d547 100644
--- a/tools/libxc/xc_psr.c
+++ b/tools/libxc/xc_psr.c
@@ -249,7 +249,7 @@ int xc_psr_cmt_enabled(xc_interface *xch)
return 0;
}
int xc_psr_cat_set_domain_data(xc_interface *xch, uint32_t domid,
- xc_psr_cat_type type, uint32_t target,
+ xc_psr_type type, uint32_t target,
uint64_t data)
{
DECLARE_DOMCTL;
@@ -284,7 +284,7 @@ int xc_psr_cat_set_domain_data(xc_interface *xch, uint32_t domid,
}
int xc_psr_cat_get_domain_data(xc_interface *xch, uint32_t domid,
- xc_psr_cat_type type, uint32_t target,
+ xc_psr_type type, uint32_t target,
uint64_t *data)
{
int rc;
diff --git a/tools/libxl/libxl_psr.c b/tools/libxl/libxl_psr.c
index 570d21a..e9d36dc 100644
--- a/tools/libxl/libxl_psr.c
+++ b/tools/libxl/libxl_psr.c
@@ -303,11 +303,11 @@ out:
return rc;
}
-static inline xc_psr_cat_type libxl__psr_cbm_type_to_libxc_psr_cat_type(
+static inline xc_psr_type libxl__psr_cbm_type_to_libxc_psr_type(
libxl_psr_cbm_type type)
{
- BUILD_BUG_ON(sizeof(libxl_psr_cbm_type) != sizeof(xc_psr_cat_type));
- return (xc_psr_cat_type)type;
+ BUILD_BUG_ON(sizeof(libxl_psr_cbm_type) != sizeof(xc_psr_type));
+ return (xc_psr_type)type;
}
int libxl_psr_cat_set_cbm(libxl_ctx *ctx, uint32_t domid,
@@ -325,12 +325,11 @@ int libxl_psr_cat_set_cbm(libxl_ctx *ctx, uint32_t domid,
}
libxl_for_each_set_bit(socketid, *target_map) {
- xc_psr_cat_type xc_type;
+ xc_psr_type xc_type = libxl__psr_cbm_type_to_libxc_psr_type(type);;
if (socketid >= nr_sockets)
break;
- xc_type = libxl__psr_cbm_type_to_libxc_psr_cat_type(type);
if (xc_psr_cat_set_domain_data(ctx->xch, domid, xc_type,
socketid, cbm)) {
libxl__psr_cat_log_err_msg(gc, errno);
@@ -349,7 +348,7 @@ int libxl_psr_cat_get_cbm(libxl_ctx *ctx, uint32_t domid,
{
GC_INIT(ctx);
int rc = 0;
- xc_psr_cat_type xc_type = libxl__psr_cbm_type_to_libxc_psr_cat_type(type);
+ xc_psr_type xc_type = libxl__psr_cbm_type_to_libxc_psr_type(type);
if (xc_psr_cat_get_domain_data(ctx->xch, domid, xc_type,
target, cbm_r)) {
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v4 13/15] tools: implement new generic get value interface and MBA get value command
2017-09-23 9:48 [PATCH v4 00/15] Enable Memory Bandwidth Allocation in Xen Yi Sun
` (11 preceding siblings ...)
2017-09-23 9:48 ` [PATCH v4 12/15] tools: rename 'xc_psr_cat_type' to 'xc_psr_type' Yi Sun
@ 2017-09-23 9:48 ` Yi Sun
2017-09-26 11:34 ` Roger Pau Monné
2017-09-23 9:48 ` [PATCH v4 14/15] tools: implement new generic set value interface and MBA set " Yi Sun
` (3 subsequent siblings)
16 siblings, 1 reply; 64+ messages in thread
From: Yi Sun @ 2017-09-23 9:48 UTC (permalink / raw)
To: xen-devel; +Cc: Wei Liu, Chao Peng, Yi Sun, Ian Jackson, Roger Pau Monné
This patch implements generic get value interfaces in libxc and libxl.
It also refactors the get value flow in xl to make it be suitable for all
allocation features. Based on that, a new MBA get value command is added in xl.
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
---
CC: Wei Liu <wei.liu2@citrix.com>
CC: Ian Jackson <ian.jackson@eu.citrix.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Chao Peng <chao.p.peng@linux.intel.com>
v4:
- use designated initializers for 'feat_name[]'.
(suggested by Roger Pau Monné)
- use LOG in 'libxl__psr_alloc_log_err_msg'.
(suggested by Roger Pau Monné)
v3:
- replace 'libxl_psr_cbm_type' to 'libxl_psr_type' in newly defined
interfaces.
(suggested by Roger Pau Monné)
v2:
- change 'CAT_INFO'/'MBA_INFO' to 'CAT'/'MBA'. The related structure names
are changed too.
(suggested by Chao Peng)
---
tools/libxc/include/xenctrl.h | 7 +-
tools/libxc/xc_psr.c | 9 +-
tools/libxl/libxl_psr.c | 58 ++++++++-----
tools/xl/xl.h | 1 +
tools/xl/xl_cmdtable.c | 5 ++
tools/xl/xl_psr.c | 185 ++++++++++++++++++++++++++++++------------
6 files changed, 183 insertions(+), 82 deletions(-)
diff --git a/tools/libxc/include/xenctrl.h b/tools/libxc/include/xenctrl.h
index 1cc4d7e..745b2a5 100644
--- a/tools/libxc/include/xenctrl.h
+++ b/tools/libxc/include/xenctrl.h
@@ -2456,6 +2456,7 @@ enum xc_psr_type {
XC_PSR_CAT_L3_CBM_CODE = 2,
XC_PSR_CAT_L3_CBM_DATA = 3,
XC_PSR_CAT_L2_CBM = 4,
+ XC_PSR_MBA_THRTL = 5,
};
typedef enum xc_psr_type xc_psr_type;
@@ -2501,9 +2502,9 @@ int xc_psr_cmt_enabled(xc_interface *xch);
int xc_psr_cat_set_domain_data(xc_interface *xch, uint32_t domid,
xc_psr_type type, uint32_t target,
uint64_t data);
-int xc_psr_cat_get_domain_data(xc_interface *xch, uint32_t domid,
- xc_psr_type type, uint32_t target,
- uint64_t *data);
+int xc_psr_get_domain_data(xc_interface *xch, uint32_t domid,
+ xc_psr_type type, uint32_t target,
+ uint64_t *data);
int xc_psr_get_hw_info(xc_interface *xch, uint32_t socket,
xc_psr_feat_type type, xc_psr_hw_info *hw_info);
diff --git a/tools/libxc/xc_psr.c b/tools/libxc/xc_psr.c
index 385d547..5660d51 100644
--- a/tools/libxc/xc_psr.c
+++ b/tools/libxc/xc_psr.c
@@ -283,9 +283,9 @@ int xc_psr_cat_set_domain_data(xc_interface *xch, uint32_t domid,
return do_domctl(xch, &domctl);
}
-int xc_psr_cat_get_domain_data(xc_interface *xch, uint32_t domid,
- xc_psr_type type, uint32_t target,
- uint64_t *data)
+int xc_psr_get_domain_data(xc_interface *xch, uint32_t domid,
+ xc_psr_type type, uint32_t target,
+ uint64_t *data)
{
int rc;
DECLARE_DOMCTL;
@@ -305,6 +305,9 @@ int xc_psr_cat_get_domain_data(xc_interface *xch, uint32_t domid,
case XC_PSR_CAT_L2_CBM:
cmd = XEN_DOMCTL_PSR_GET_L2_CBM;
break;
+ case XC_PSR_MBA_THRTL:
+ cmd = XEN_DOMCTL_PSR_GET_MBA_THRTL;
+ break;
default:
errno = EINVAL;
return -1;
diff --git a/tools/libxl/libxl_psr.c b/tools/libxl/libxl_psr.c
index e9d36dc..5a4ba3d 100644
--- a/tools/libxl/libxl_psr.c
+++ b/tools/libxl/libxl_psr.c
@@ -71,16 +71,29 @@ static void libxl__psr_cmt_log_err_msg(libxl__gc *gc, int err)
LOGE(ERROR, "%s", msg);
}
-static void libxl__psr_cat_log_err_msg(libxl__gc *gc, int err)
+static void libxl__psr_alloc_log_err_msg(libxl__gc *gc,
+ int err,
+ libxl_psr_type type)
{
+ /*
+ * Index is 'libxl_psr_type' so we set two 'CDP' to correspond to
+ * DATA and CODE.
+ */
+ const char * const feat_name[] = {
+ [LIBXL_PSR_CBM_TYPE_UNKNOWN] = "UNKNOWN",
+ [LIBXL_PSR_CBM_TYPE_L3_CBM] = "L3 CAT",
+ [LIBXL_PSR_CBM_TYPE_L3_CBM_CODE...LIBXL_PSR_CBM_TYPE_L3_CBM_DATA] = "CDP",
+ [LIBXL_PSR_CBM_TYPE_L2_CBM] = "L2 CAT",
+ [LIBXL_PSR_CBM_TYPE_MBA_THRTL] = "MBA",
+ };
char *msg;
switch (err) {
case ENODEV:
- msg = "CAT is not supported in this system";
+ msg = "is not supported in this system";
break;
case ENOENT:
- msg = "CAT is not enabled on the socket";
+ msg = "is not enabled on the socket";
break;
case EOVERFLOW:
msg = "no free COS available";
@@ -106,7 +119,7 @@ static void libxl__psr_cat_log_err_msg(libxl__gc *gc, int err)
return;
}
- LOGE(ERROR, "%s", msg);
+ LOG(ERROR, "%s: %s", feat_name[type], msg);
}
static int libxl__pick_socket_cpu(libxl__gc *gc, uint32_t socketid)
@@ -303,10 +316,10 @@ out:
return rc;
}
-static inline xc_psr_type libxl__psr_cbm_type_to_libxc_psr_type(
- libxl_psr_cbm_type type)
+static inline xc_psr_type libxl__psr_type_to_libxc_psr_type(
+ libxl_psr_type type)
{
- BUILD_BUG_ON(sizeof(libxl_psr_cbm_type) != sizeof(xc_psr_type));
+ BUILD_BUG_ON(sizeof(libxl_psr_type) != sizeof(xc_psr_type));
return (xc_psr_type)type;
}
@@ -325,14 +338,14 @@ int libxl_psr_cat_set_cbm(libxl_ctx *ctx, uint32_t domid,
}
libxl_for_each_set_bit(socketid, *target_map) {
- xc_psr_type xc_type = libxl__psr_cbm_type_to_libxc_psr_type(type);;
+ xc_psr_type xc_type = libxl__psr_type_to_libxc_psr_type(type);;
if (socketid >= nr_sockets)
break;
if (xc_psr_cat_set_domain_data(ctx->xch, domid, xc_type,
socketid, cbm)) {
- libxl__psr_cat_log_err_msg(gc, errno);
+ libxl__psr_alloc_log_err_msg(gc, errno, type);
rc = ERROR_FAIL;
}
}
@@ -346,18 +359,7 @@ int libxl_psr_cat_get_cbm(libxl_ctx *ctx, uint32_t domid,
libxl_psr_cbm_type type, uint32_t target,
uint64_t *cbm_r)
{
- GC_INIT(ctx);
- int rc = 0;
- xc_psr_type xc_type = libxl__psr_cbm_type_to_libxc_psr_type(type);
-
- if (xc_psr_cat_get_domain_data(ctx->xch, domid, xc_type,
- target, cbm_r)) {
- libxl__psr_cat_log_err_msg(gc, errno);
- rc = ERROR_FAIL;
- }
-
- GC_FREE;
- return rc;
+ return libxl_psr_get_val(ctx, domid, type, target, cbm_r);
}
static xc_psr_feat_type libxl__feat_type_to_libxc_feat_type(
@@ -455,7 +457,19 @@ int libxl_psr_get_val(libxl_ctx *ctx, uint32_t domid,
libxl_psr_type type, unsigned int target,
uint64_t *val)
{
- return ERROR_FAIL;
+ GC_INIT(ctx);
+ int rc = 0;
+
+ xc_psr_type xc_type = libxl__psr_type_to_libxc_psr_type(type);
+
+ if (xc_psr_get_domain_data(ctx->xch, domid, xc_type,
+ target, val)) {
+ libxl__psr_alloc_log_err_msg(gc, errno, type);
+ rc = ERROR_FAIL;
+ }
+
+ GC_FREE;
+ return rc;
}
static void libxl__xc_hw_info_to_libxl_hw_info(
diff --git a/tools/xl/xl.h b/tools/xl/xl.h
index 01c2af6..464831e 100644
--- a/tools/xl/xl.h
+++ b/tools/xl/xl.h
@@ -205,6 +205,7 @@ int main_psr_cmt_detach(int argc, char **argv);
int main_psr_cmt_show(int argc, char **argv);
int main_psr_cat_cbm_set(int argc, char **argv);
int main_psr_cat_show(int argc, char **argv);
+int main_psr_mba_show(int argc, char **argv);
#endif
int main_qemu_monitor_command(int argc, char **argv);
diff --git a/tools/xl/xl_cmdtable.c b/tools/xl/xl_cmdtable.c
index a1679e9..3136d70 100644
--- a/tools/xl/xl_cmdtable.c
+++ b/tools/xl/xl_cmdtable.c
@@ -565,6 +565,11 @@ struct cmd_spec cmd_table[] = {
"[options] <Domain>",
"-l <level> Specify the cache level to process, otherwise L3 cache is processed\n"
},
+ { "psr-mba-show",
+ &main_psr_mba_show, 0, 1,
+ "Show Memory Bandwidth Allocation information",
+ "<Domain>",
+ },
#endif
{ "usbctrl-attach",
&main_usbctrl_attach, 0, 1,
diff --git a/tools/xl/xl_psr.c b/tools/xl/xl_psr.c
index 40269b4..46b7788 100644
--- a/tools/xl/xl_psr.c
+++ b/tools/xl/xl_psr.c
@@ -327,19 +327,27 @@ out:
return rc;
}
-static void psr_cat_print_one_domain_cbm_type(uint32_t domid, uint32_t socketid,
- libxl_psr_cbm_type type)
+static void psr_print_one_domain_val_type(uint32_t domid,
+ libxl_psr_hw_info *info,
+ libxl_psr_type type)
{
- uint64_t cbm;
+ uint64_t val;
- if (!libxl_psr_cat_get_cbm(ctx, domid, type, socketid, &cbm))
- printf("%#16"PRIx64, cbm);
+ if (!libxl_psr_get_val(ctx, domid, type, info->id, &val))
+ {
+ if (type == LIBXL_PSR_CBM_TYPE_MBA_THRTL && info->u.mba.linear)
+ printf("%16"PRIu64, val);
+ else
+ printf("%#16"PRIx64, val);
+ }
else
printf("%16s", "error");
}
-static void psr_cat_print_one_domain_cbm(uint32_t domid, uint32_t socketid,
- bool cdp_enabled, unsigned int lvl)
+static void psr_print_one_domain_val(uint32_t domid,
+ libxl_psr_hw_info *info,
+ libxl_psr_feat_type type,
+ unsigned int lvl)
{
char *domain_name;
@@ -347,106 +355,154 @@ static void psr_cat_print_one_domain_cbm(uint32_t domid, uint32_t socketid,
printf("%5d%25s", domid, domain_name);
free(domain_name);
- switch (lvl) {
- case 3:
- if (!cdp_enabled) {
- psr_cat_print_one_domain_cbm_type(domid, socketid,
+ switch (type) {
+ case LIBXL_PSR_FEAT_TYPE_CAT:
+ switch (lvl) {
+ case 3:
+ if (!info->u.cat.cdp_enabled) {
+ psr_print_one_domain_val_type(domid, info,
LIBXL_PSR_CBM_TYPE_L3_CBM);
- } else {
- psr_cat_print_one_domain_cbm_type(domid, socketid,
+ } else {
+ psr_print_one_domain_val_type(domid, info,
LIBXL_PSR_CBM_TYPE_L3_CBM_CODE);
- psr_cat_print_one_domain_cbm_type(domid, socketid,
+ psr_print_one_domain_val_type(domid, info,
LIBXL_PSR_CBM_TYPE_L3_CBM_DATA);
- }
- break;
- case 2:
- psr_cat_print_one_domain_cbm_type(domid, socketid,
+ }
+ break;
+
+ case 2:
+ psr_print_one_domain_val_type(domid, info,
LIBXL_PSR_CBM_TYPE_L2_CBM);
+ break;
+
+ default:
+ printf("Input lvl %d is wrong!", lvl);
+ }
break;
- default:
- printf("Input lvl %d is wrong!", lvl);
+
+ case LIBXL_PSR_FEAT_TYPE_MBA:
+ psr_print_one_domain_val_type(domid, info,
+ LIBXL_PSR_CBM_TYPE_MBA_THRTL);
break;
}
printf("\n");
}
-static int psr_cat_print_domain_cbm(uint32_t domid, uint32_t socketid,
- bool cdp_enabled, unsigned int lvl)
+static int psr_print_domain_val(uint32_t domid,
+ libxl_psr_hw_info *info,
+ libxl_psr_feat_type type,
+ unsigned int lvl)
{
int i, nr_domains;
libxl_dominfo *list;
if (domid != INVALID_DOMID) {
- psr_cat_print_one_domain_cbm(domid, socketid, cdp_enabled, lvl);
+ psr_print_one_domain_val(domid, info, type, lvl);
return 0;
}
if (!(list = libxl_list_domain(ctx, &nr_domains))) {
- fprintf(stderr, "Failed to get domain list for cbm display\n");
- return -1;
+ fprintf(stderr, "Failed to get domain list for value display\n");
+ return EXIT_FAILURE;
}
for (i = 0; i < nr_domains; i++)
- psr_cat_print_one_domain_cbm(list[i].domid, socketid, cdp_enabled, lvl);
+ psr_print_one_domain_val(list[i].domid, info, type, lvl);
libxl_dominfo_list_free(list, nr_domains);
return 0;
}
-static int psr_cat_print_socket(uint32_t domid, libxl_psr_cat_info *info,
- unsigned int lvl)
+static int psr_print_socket(uint32_t domid,
+ libxl_psr_hw_info *info,
+ libxl_psr_feat_type type,
+ unsigned int lvl)
{
- int rc;
- uint32_t l3_cache_size;
-
printf("%-16s: %u\n", "Socket ID", info->id);
- /* So far, CMT only supports L3 cache. */
- if (lvl == 3) {
- rc = libxl_psr_cmt_get_l3_cache_size(ctx, info->id, &l3_cache_size);
- if (rc) {
- fprintf(stderr, "Failed to get l3 cache size for socket:%d\n",
- info->id);
- return -1;
+ switch (type) {
+ case LIBXL_PSR_FEAT_TYPE_CAT:
+ {
+ int rc;
+ uint32_t l3_cache_size;
+
+ /* So far, CMT only supports L3 cache. */
+ if (lvl == 3) {
+ rc = libxl_psr_cmt_get_l3_cache_size(ctx, info->id, &l3_cache_size);
+ if (rc) {
+ fprintf(stderr, "Failed to get l3 cache size for socket:%d\n",
+ info->id);
+ return -1;
+ }
+ printf("%-16s: %uKB\n", "L3 Cache", l3_cache_size);
}
- printf("%-16s: %uKB\n", "L3 Cache", l3_cache_size);
+
+ printf("%-16s: %#llx\n", "Default CBM",
+ (1ull << info->u.cat.cbm_len) - 1);
+ if (info->u.cat.cdp_enabled)
+ printf("%5s%25s%16s%16s\n", "ID", "NAME", "CBM (code)", "CBM (data)");
+ else
+ printf("%5s%25s%16s\n", "ID", "NAME", "CBM");
+
+ break;
}
- printf("%-16s: %#llx\n", "Default CBM", (1ull << info->cbm_len) - 1);
- if (info->cdp_enabled)
- printf("%5s%25s%16s%16s\n", "ID", "NAME", "CBM (code)", "CBM (data)");
- else
- printf("%5s%25s%16s\n", "ID", "NAME", "CBM");
+ case LIBXL_PSR_FEAT_TYPE_MBA:
+ printf("%-16s: %u\n", "Default THRTL", 0);
+ printf("%5s%25s%16s\n", "ID", "NAME", "THRTL");
+ break;
- return psr_cat_print_domain_cbm(domid, info->id, info->cdp_enabled, lvl);
+ default:
+ fprintf(stderr, "Input feature type %d is wrong\n", type);
+ return EXIT_FAILURE;
+ }
+
+ return psr_print_domain_val(domid, info, type, lvl);
}
-static int psr_cat_show(uint32_t domid, unsigned int lvl)
+static int psr_val_show(uint32_t domid,
+ libxl_psr_feat_type type,
+ unsigned int lvl)
{
unsigned int i, nr;
int rc;
- libxl_psr_cat_info *info;
+ libxl_psr_hw_info *info;
- if (lvl != 2 && lvl != 3) {
- fprintf(stderr, "Input lvl %d is wrong\n", lvl);
+ switch (type) {
+ case LIBXL_PSR_FEAT_TYPE_CAT:
+ if (lvl != 2 && lvl != 3) {
+ fprintf(stderr, "Input lvl %d is wrong\n", lvl);
+ return EXIT_FAILURE;
+ }
+ break;
+
+ case LIBXL_PSR_FEAT_TYPE_MBA:
+ if (lvl) {
+ fprintf(stderr, "Input lvl %d is wrong\n", lvl);
+ return EXIT_FAILURE;
+ }
+ break;
+
+ default:
+ fprintf(stderr, "Input feature type %d is wrong\n", type);
return EXIT_FAILURE;
}
- rc = libxl_psr_cat_get_info(ctx, &info, &nr, lvl);
+ rc = libxl_psr_get_hw_info(ctx, &info, &nr, type, lvl);
if (rc) {
- fprintf(stderr, "Failed to get %s cat info\n", (lvl == 3)?"L3":"L2");
+ fprintf(stderr, "Failed to get info\n");
return rc;
}
for (i = 0; i < nr; i++) {
- rc = psr_cat_print_socket(domid, info + i, lvl);
+ rc = psr_print_socket(domid, info + i, type, lvl);
if (rc)
goto out;
}
out:
- libxl_psr_cat_info_list_free(info, nr);
+ libxl_psr_hw_info_list_free(info, nr);
return rc;
}
@@ -475,6 +531,27 @@ static int psr_l2_cat_hwinfo(void)
return rc;
}
+int main_psr_mba_show(int argc, char **argv)
+{
+ int opt;
+ uint32_t domid;
+
+ SWITCH_FOREACH_OPT(opt, "", NULL, "psr-mba-show", 0) {
+ /* No options */
+ }
+
+ if (optind >= argc)
+ domid = INVALID_DOMID;
+ else if (optind == argc - 1)
+ domid = find_domain(argv[optind]);
+ else {
+ help("psr-mba-show");
+ return 2;
+ }
+
+ return psr_val_show(domid, LIBXL_PSR_FEAT_TYPE_MBA, 0);
+}
+
static int psr_mba_hwinfo(void)
{
int rc;
@@ -613,7 +690,7 @@ int main_psr_cat_show(int argc, char **argv)
return 2;
}
- return psr_cat_show(domid, lvl);
+ return psr_val_show(domid, LIBXL_PSR_FEAT_TYPE_CAT, lvl);
}
int main_psr_hwinfo(int argc, char **argv)
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v4 14/15] tools: implement new generic set value interface and MBA set value command
2017-09-23 9:48 [PATCH v4 00/15] Enable Memory Bandwidth Allocation in Xen Yi Sun
` (12 preceding siblings ...)
2017-09-23 9:48 ` [PATCH v4 13/15] tools: implement new generic get value interface and MBA get value command Yi Sun
@ 2017-09-23 9:48 ` Yi Sun
2017-09-26 11:39 ` Roger Pau Monné
2017-09-23 9:48 ` [PATCH v4 15/15] docs: add MBA description in docs Yi Sun
` (2 subsequent siblings)
16 siblings, 1 reply; 64+ messages in thread
From: Yi Sun @ 2017-09-23 9:48 UTC (permalink / raw)
To: xen-devel; +Cc: Wei Liu, Chao Peng, Yi Sun, Ian Jackson, Roger Pau Monné
This patch implements new generic set value interfaces in libxc and libxl.
These interfaces are suitable for all allocation features. It also adds a
new MBA set value command in xl.
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
---
CC: Wei Liu <wei.liu2@citrix.com>
CC: Ian Jackson <ian.jackson@eu.citrix.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Chao Peng <chao.p.peng@linux.intel.com>
v4:
- remove 'ALLOC_' from macro name.
(suggested by Roger Pau Monné)
- adjust place of argc check and return EXIT_FAILURE.
(suggested by Roger Pau Monné)
- fix indentation issue.
(suggested by Roger Pau Monné)
- move same type local variables declaration to a single line.
(suggested by Roger Pau Monné)
v3:
- add 'const' for 'opts[]' in 'main_psr_mba_set'.
(suggested by Roger Pau Monné)
- replace 'libxl_psr_cbm_type' to 'libxl_psr_type' for newly defined
interfaces.
(suggested by Roger Pau Monné)
---
tools/libxc/include/xenctrl.h | 6 ++---
tools/libxc/xc_psr.c | 9 ++++---
tools/libxl/libxl_psr.c | 54 +++++++++++++++++++++---------------------
tools/xl/xl.h | 1 +
tools/xl/xl_cmdtable.c | 6 +++++
tools/xl/xl_psr.c | 55 +++++++++++++++++++++++++++++++++++++++++++
6 files changed, 98 insertions(+), 33 deletions(-)
diff --git a/tools/libxc/include/xenctrl.h b/tools/libxc/include/xenctrl.h
index 745b2a5..ae6a436 100644
--- a/tools/libxc/include/xenctrl.h
+++ b/tools/libxc/include/xenctrl.h
@@ -2499,9 +2499,9 @@ int xc_psr_cmt_get_data(xc_interface *xch, uint32_t rmid, uint32_t cpu,
uint64_t *tsc);
int xc_psr_cmt_enabled(xc_interface *xch);
-int xc_psr_cat_set_domain_data(xc_interface *xch, uint32_t domid,
- xc_psr_type type, uint32_t target,
- uint64_t data);
+int xc_psr_set_domain_data(xc_interface *xch, uint32_t domid,
+ xc_psr_type type, uint32_t target,
+ uint64_t data);
int xc_psr_get_domain_data(xc_interface *xch, uint32_t domid,
xc_psr_type type, uint32_t target,
uint64_t *data);
diff --git a/tools/libxc/xc_psr.c b/tools/libxc/xc_psr.c
index 5660d51..de9ca82 100644
--- a/tools/libxc/xc_psr.c
+++ b/tools/libxc/xc_psr.c
@@ -248,9 +248,9 @@ int xc_psr_cmt_enabled(xc_interface *xch)
return 0;
}
-int xc_psr_cat_set_domain_data(xc_interface *xch, uint32_t domid,
- xc_psr_type type, uint32_t target,
- uint64_t data)
+int xc_psr_set_domain_data(xc_interface *xch, uint32_t domid,
+ xc_psr_type type, uint32_t target,
+ uint64_t data)
{
DECLARE_DOMCTL;
uint32_t cmd;
@@ -269,6 +269,9 @@ int xc_psr_cat_set_domain_data(xc_interface *xch, uint32_t domid,
case XC_PSR_CAT_L2_CBM:
cmd = XEN_DOMCTL_PSR_SET_L2_CBM;
break;
+ case XC_PSR_MBA_THRTL:
+ cmd = XEN_DOMCTL_PSR_SET_MBA_THRTL;
+ break;
default:
errno = EINVAL;
return -1;
diff --git a/tools/libxl/libxl_psr.c b/tools/libxl/libxl_psr.c
index 5a4ba3d..88fa39c 100644
--- a/tools/libxl/libxl_psr.c
+++ b/tools/libxl/libxl_psr.c
@@ -327,32 +327,7 @@ int libxl_psr_cat_set_cbm(libxl_ctx *ctx, uint32_t domid,
libxl_psr_cbm_type type, libxl_bitmap *target_map,
uint64_t cbm)
{
- GC_INIT(ctx);
- int rc;
- int socketid, nr_sockets;
-
- rc = libxl__count_physical_sockets(gc, &nr_sockets);
- if (rc) {
- LOGED(ERROR, domid, "failed to get system socket count");
- goto out;
- }
-
- libxl_for_each_set_bit(socketid, *target_map) {
- xc_psr_type xc_type = libxl__psr_type_to_libxc_psr_type(type);;
-
- if (socketid >= nr_sockets)
- break;
-
- if (xc_psr_cat_set_domain_data(ctx->xch, domid, xc_type,
- socketid, cbm)) {
- libxl__psr_alloc_log_err_msg(gc, errno, type);
- rc = ERROR_FAIL;
- }
- }
-
-out:
- GC_FREE;
- return rc;
+ return libxl_psr_set_val(ctx, domid, type, target_map, cbm);
}
int libxl_psr_cat_get_cbm(libxl_ctx *ctx, uint32_t domid,
@@ -450,7 +425,32 @@ int libxl_psr_set_val(libxl_ctx *ctx, uint32_t domid,
libxl_psr_type type, libxl_bitmap *target_map,
uint64_t val)
{
- return ERROR_FAIL;
+ GC_INIT(ctx);
+ int rc, socketid, nr_sockets;
+
+ rc = libxl__count_physical_sockets(gc, &nr_sockets);
+ if (rc) {
+ LOG(ERROR, "failed to get system socket count");
+ goto out;
+ }
+
+ libxl_for_each_set_bit(socketid, *target_map) {
+ xc_psr_type xc_type;
+
+ if (socketid >= nr_sockets)
+ break;
+
+ xc_type = libxl__psr_type_to_libxc_psr_type(type);
+ if (xc_psr_set_domain_data(ctx->xch, domid, xc_type,
+ socketid, val)) {
+ libxl__psr_alloc_log_err_msg(gc, errno, type);
+ rc = ERROR_FAIL;
+ }
+ }
+
+out:
+ GC_FREE;
+ return rc;
}
int libxl_psr_get_val(libxl_ctx *ctx, uint32_t domid,
diff --git a/tools/xl/xl.h b/tools/xl/xl.h
index 464831e..89309e7 100644
--- a/tools/xl/xl.h
+++ b/tools/xl/xl.h
@@ -205,6 +205,7 @@ int main_psr_cmt_detach(int argc, char **argv);
int main_psr_cmt_show(int argc, char **argv);
int main_psr_cat_cbm_set(int argc, char **argv);
int main_psr_cat_show(int argc, char **argv);
+int main_psr_mba_set(int argc, char **argv);
int main_psr_mba_show(int argc, char **argv);
#endif
int main_qemu_monitor_command(int argc, char **argv);
diff --git a/tools/xl/xl_cmdtable.c b/tools/xl/xl_cmdtable.c
index 3136d70..fb21317 100644
--- a/tools/xl/xl_cmdtable.c
+++ b/tools/xl/xl_cmdtable.c
@@ -565,6 +565,12 @@ struct cmd_spec cmd_table[] = {
"[options] <Domain>",
"-l <level> Specify the cache level to process, otherwise L3 cache is processed\n"
},
+ { "psr-mba-set",
+ &main_psr_mba_set, 0, 1,
+ "Set throttling value (THRTL) for a domain",
+ "[options] <Domain> <THRTL>",
+ "-s <socket> Specify the socket to process, otherwise all sockets are processed\n"
+ },
{ "psr-mba-show",
&main_psr_mba_show, 0, 1,
"Show Memory Bandwidth Allocation information",
diff --git a/tools/xl/xl_psr.c b/tools/xl/xl_psr.c
index 46b7788..60aa628 100644
--- a/tools/xl/xl_psr.c
+++ b/tools/xl/xl_psr.c
@@ -552,6 +552,61 @@ int main_psr_mba_show(int argc, char **argv)
return psr_val_show(domid, LIBXL_PSR_FEAT_TYPE_MBA, 0);
}
+int main_psr_mba_set(int argc, char **argv)
+{
+ uint32_t domid;
+ libxl_psr_type type;
+ uint64_t thrtl;
+ int ret, opt = 0;
+ libxl_bitmap target_map;
+ char *value;
+ libxl_string_list socket_list;
+ unsigned long start, end;
+ unsigned int i, j, len;
+
+ static const struct option opts[] = {
+ {"socket", 1, 0, 's'},
+ COMMON_LONG_OPTS
+ };
+
+ if (argc != optind + 2) {
+ help("psr-mba-set");
+ return EXIT_FAILURE;
+ }
+
+ libxl_socket_bitmap_alloc(ctx, &target_map, 0);
+ libxl_bitmap_set_none(&target_map);
+
+ SWITCH_FOREACH_OPT(opt, "s:", opts, "psr-mba-set", 0) {
+ case 's':
+ trim(isspace, optarg, &value);
+ split_string_into_string_list(value, ",", &socket_list);
+ len = libxl_string_list_length(&socket_list);
+ for (i = 0; i < len; i++) {
+ parse_range(socket_list[i], &start, &end);
+ for (j = start; j <= end; j++)
+ libxl_bitmap_set(&target_map, j);
+ }
+
+ libxl_string_list_dispose(&socket_list);
+ free(value);
+ break;
+ }
+
+ type = LIBXL_PSR_CBM_TYPE_MBA_THRTL;
+
+ if (libxl_bitmap_is_empty(&target_map))
+ libxl_bitmap_set_any(&target_map);
+
+ domid = find_domain(argv[optind]);
+ thrtl = strtoll(argv[optind + 1], NULL , 0);
+
+ ret = libxl_psr_set_val(ctx, domid, type, &target_map, thrtl);
+
+ libxl_bitmap_dispose(&target_map);
+ return ret;
+}
+
static int psr_mba_hwinfo(void)
{
int rc;
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v4 15/15] docs: add MBA description in docs
2017-09-23 9:48 [PATCH v4 00/15] Enable Memory Bandwidth Allocation in Xen Yi Sun
` (13 preceding siblings ...)
2017-09-23 9:48 ` [PATCH v4 14/15] tools: implement new generic set value interface and MBA set " Yi Sun
@ 2017-09-23 9:48 ` Yi Sun
2017-09-26 11:45 ` Roger Pau Monné
2017-09-26 11:48 ` [PATCH v4 00/15] Enable Memory Bandwidth Allocation in Xen Roger Pau Monné
2017-09-28 15:57 ` Wei Liu
16 siblings, 1 reply; 64+ messages in thread
From: Yi Sun @ 2017-09-23 9:48 UTC (permalink / raw)
To: xen-devel; +Cc: Wei Liu, Chao Peng, Yi Sun, Ian Jackson, Roger Pau Monné
This patch adds MBA description in related documents.
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
---
CC: Wei Liu <wei.liu2@citrix.com>
CC: Ian Jackson <ian.jackson@eu.citrix.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Chao Peng <chao.p.peng@linux.intel.com>
v4:
- modify description of MBA in 'xl.pod.1.in' to be same as feature doc.
(suggested by Roger Pau Monné)
- fix words issue.
(suggested by Roger Pau Monné)
v2:
- state the value type shown by 'psr-mba-show'. For linear mode,
it shows decimal value. For non-linear mode, it shows hexadecimal
value.
(suggested by Chao Peng)
---
docs/man/xl.pod.1.in | 33 +++++++++++++++++++++++++
docs/misc/xl-psr.markdown | 63 +++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 96 insertions(+)
diff --git a/docs/man/xl.pod.1.in b/docs/man/xl.pod.1.in
index 3d5f2f7..64d87ba 100644
--- a/docs/man/xl.pod.1.in
+++ b/docs/man/xl.pod.1.in
@@ -1803,6 +1803,39 @@ processed.
=back
+=head2 Memory Bandwidth Allocation
+
+Intel Skylake and later server platforms offer capabilities to configure and
+make use of the Memory Bandwidth Allocation (MBA) mechanisms, which provides
+OS/VMMs the ability to slow misbehaving apps/VMs by using a credit-based
+throttling mechanism. In the Xen implementation, MBA is used to control memory
+bandwidth on VM basis. To enforce bandwidth on a specific domain, just set
+throttling value (THRTL) for the domain.
+
+=over 4
+
+=item B<psr-mba-set> [I<OPTIONS>] I<domain-id> I<thrtl>
+
+Set throttling value (THRTL) for a domain. For how to specify I<thrtl>
+please refer to L<http://xenbits.xen.org/docs/unstable/misc/xl-psr.html>.
+
+B<OPTIONS>
+
+=over 4
+
+=item B<-s SOCKET>, B<--socket=SOCKET>
+
+Specify the socket to process, otherwise all sockets are processed.
+
+=back
+
+=item B<psr-mba-show> [I<domain-id>]
+
+Show MBA settings for a certain domain or all domains. For linear mode, it
+shows the decimal value. For non-linear mode, it shows hexadecimal value.
+
+=back
+
=head1 IGNORED FOR COMPATIBILITY WITH XM
xl is mostly command-line compatible with the old xm utility used with
diff --git a/docs/misc/xl-psr.markdown b/docs/misc/xl-psr.markdown
index 04dd957..08c06dc 100644
--- a/docs/misc/xl-psr.markdown
+++ b/docs/misc/xl-psr.markdown
@@ -186,6 +186,69 @@ Setting data CBM for a domain:
Setting the same code and data CBM for a domain:
`xl psr-cat-set <domid> <cbm>`
+## Memory Bandwidth Allocation (MBA)
+
+Memory Bandwidth Allocation (MBA) is a new feature available on Intel
+Skylake and later server platforms that allows an OS or Hypervisor/VMM to
+slow misbehaving apps/VMs or create advanced closed-loop control system via
+exposing control over a credit-based throttling mechanism. To enforce bandwidth
+on a specific domain, just set throttling value (THRTL) into Class of Service
+(COS). MBA provides two THRTL mode. One is linear mode and the other is
+non-linear mode.
+
+In the linear mode the input precision is defined as 100-(THRTL_MAX). Values
+not an even multiple of the precision (e.g., 12%) will be rounded down (e.g.,
+to 10% delay by the hardware).
+
+If linear values are not supported then input delay values are powers-of-two
+from zero to the THRTL_MAX value from CPUID. In this case any values not a power
+of two will be rounded down the next nearest power of two.
+
+For example, assuming a system with 2 domains:
+
+ * A THRTL of 0x0 for every domain means each domain can access the whole cache
+ without any delay. This is the default.
+
+ * Linear mode: Giving one domain a THRTL of 0xC and the other domain's 0 means
+ that the first domain gets 10% delay to access the cache and the other one
+ without any delay.
+
+ * Non-linear mode: Giving one domain a THRTL of 0xC and the other domain's 0
+ means that the first domain gets 8% delay to access the cache and the other
+ one without any delay.
+
+For more detailed information please refer to Intel SDM chapter
+"Introduction to Memory Bandwidth Allocation".
+
+In Xen's implementation, THRTL can be configured with libxl/xl interfaces but
+COS is maintained in hypervisor only. The cache partition granularity is per
+domain, each domain has COS=0 assigned by default, the corresponding THRTL is
+0, which means all the cache resource can be accessed without delay.
+
+### xl interfaces
+
+System MBA information such as maximum COS and maximum THRTL can be obtained by:
+
+`xl psr-hwinfo --mba`
+
+The simplest way to change a domain's THRTL from its default is running:
+
+`xl psr-mba-set [OPTIONS] <domid> <thrtl>`
+
+In a multi-socket system, the same thrtl will be set on each socket by default.
+Per socket thrtl can be specified with the `--socket SOCKET` option.
+
+Setting the THRTL may not be successful if insufficient COS is available. In
+such case unused COS(es) may be freed by setting THRTL of all related domains to
+its default value(0).
+
+Per domain THRTL settings can be shown by:
+
+`xl psr-mba-show [OPTIONS] <domid>`
+
+For linear mode, it shows the decimal value. For non-linear mode, it shows
+hexadecimal value.
+
## Reference
[1] Intel SDM
--
1.9.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 64+ messages in thread
* Re: [PATCH v4 01/15] docs: create Memory Bandwidth Allocation (MBA) feature document
2017-09-23 9:48 ` [PATCH v4 01/15] docs: create Memory Bandwidth Allocation (MBA) feature document Yi Sun
@ 2017-09-25 15:13 ` Roger Pau Monné
0 siblings, 0 replies; 64+ messages in thread
From: Roger Pau Monné @ 2017-09-25 15:13 UTC (permalink / raw)
To: Yi Sun
Cc: Wei Liu, Andrew Cooper, Ian Jackson, Julien Grall, Jan Beulich,
Chao Peng, xen-devel, Daniel De Graaf
On Sat, Sep 23, 2017 at 09:48:10AM +0000, Yi Sun wrote:
> This patch creates MBA feature document in doc/features/. It describes
> key points to implement MBA which is described in details in Intel SDM
^ detail
> "Introduction to Memory Bandwidth Allocation".
>
> Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
Thanks, I think this is looking quite good IMHO. Just a couple of nits
below.
> ---
> CC: Jan Beulich <jbeulich@suse.com>
> CC: Andrew Cooper <andrew.cooper3@citrix.com>
> CC: Wei Liu <wei.liu2@citrix.com>
> CC: Ian Jackson <ian.jackson@eu.citrix.com>
> CC: Daniel De Graaf <dgdegra@tycho.nsa.gov>
> CC: Roger Pau Monné <roger.pau@citrix.com>
> CC: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> CC: Chao Peng <chao.p.peng@linux.intel.com>
> CC: Julien Grall <julien.grall@arm.com>
>
> v4:
> - add 'domain-name' as parameter of 'psr-mba-show/psr-mba-set'.
> (suggested by Roger Pau Monné)
> - fix some wordings.
> (suggested by Roger Pau Monné)
> - explain how user can know the MBA_MAX.
> (suggested by Roger Pau Monné)
> - move the description of 'Linear mode/Non-linear mode' into section
> of 'psr-mba-show'.
> (suggested by Roger Pau Monné)
> - change 'per-thread' to 'per-hyper-thread' to make it clearer.
> (suggested by Roger Pau Monné)
> - upgrade revision number.
> v3:
> - remove 'closed-loop' related description.
> (suggested by Roger Pau Monné)
> - explain 'linear' and 'non-linear' before mentioning them.
> (suggested by Roger Pau Monné)
> - adjust desription of 'psr-mba-set'.
> (suggested by Roger Pau Monné)
> - explain 'MBA_MAX'.
> (suggested by Roger Pau Monné)
> - remove 'n<64'.
> (suggested by Roger Pau Monné)
> - fix some wordings.
> (suggested by Roger Pau Monné)
> - add context in 'Testing' part to make things more clear.
> (suggested by Roger Pau Monné)
> v2:
> - declare 'HW' in Terminology.
> (suggested by Chao Peng)
> - replace 'COS ID of VCPU' to 'COS ID of domain'.
> (suggested by Chao Peng)
> - replace 'COS register' to 'Thrtl MSR'.
> (suggested by Chao Peng)
> - add description for 'psr-mba-show' to state that the decimal value is
> shown for linear mode but hexadecimal value is shown for non-linear mode.
> (suggested by Chao Peng)
> - remove content in 'Areas for improvement'.
> (suggested by Chao Peng)
> - use '<>' to specify mandatory argument to a command.
> (suggested by Wei Liu)
> v1:
> - remove a special character to avoid the error when building pandoc.
> ---
> docs/features/intel_psr_mba.pandoc | 291 +++++++++++++++++++++++++++++++++++++
> 1 file changed, 291 insertions(+)
> create mode 100644 docs/features/intel_psr_mba.pandoc
>
> diff --git a/docs/features/intel_psr_mba.pandoc b/docs/features/intel_psr_mba.pandoc
> new file mode 100644
> index 0000000..7a6a588
> --- /dev/null
> +++ b/docs/features/intel_psr_mba.pandoc
> @@ -0,0 +1,291 @@
> +% Intel Memory Bandwidth Allocation (MBA) Feature
> +% Revision 1.6
> +
> +\clearpage
> +
> +# Basics
> +
> +---------------- ----------------------------------------------------
> + Status: **Tech Preview**
> +
> +Architecture(s): Intel x86
> +
> + Component(s): Hypervisor, toolstack
> +
> + Hardware: MBA is supported on Skylake Server and beyond
> +---------------- ----------------------------------------------------
> +
> +# Terminology
> +
> +* CAT Cache Allocation Technology
> +* CBM Capacity BitMasks
> +* CDP Code and Data Prioritization
> +* COS/CLOS Class of Service
> +* HW Hardware
> +* MBA Memory Bandwidth Allocation
> +* MSRs Machine Specific Registers
> +* PSR Intel Platform Shared Resource
> +* THRTL Throttle value or delay value
> +
> +# Overview
> +
> +The Memory Bandwidth Allocation (MBA) feature provides indirect and approximate
> +control over memory bandwidth available per-core. This feature provides OS/
> +hypervisor the ability to slow misbehaving apps/domains by using a credit-based
> +throttling mechanism.
> +
> +# User details
> +
> +* Feature Enabling:
> +
> + Add "psr=mba" to boot line parameter to enable MBA feature.
> +
> +* xl interfaces:
> +
> + 1. `psr-mba-show [domain-id|domain-name]`:
> +
> + Show memory bandwidth throttling for domain. Under different modes, it
> + shows different type of data.
> +
> + There are two modes:
> + Linear mode: the input precision is defined as 100-(MBA_MAX). For instance,
> + if the MBA_MAX value is 90, the input precision is 10%. Values not an even
> + multiple of the precision (e.g., 12%) will be rounded down (e.g., to 10%
> + delay applied) by HW automatically. The response of throttling value is
> + linear.
> +
> + Non-linear mode: input delay values are powers-of-two from zero to the
> + MBA_MAX value from CPUID. In this case any values not a power of two will
> + be rounded down the next nearest power of two by HW automatically. The
> + response of throttling value is non-linear.
> +
> + For linear mode, it shows the decimal value. For non-linear mode, it shows
> + hexadecimal value.
> +
> + 2. `psr-mba-set [OPTIONS] <domain-id|domain-name> <throttling>`:
> +
> + Set memory bandwidth throttling for domain.
> +
> + Options:
> + '-s': Specify the socket to process, otherwise all sockets are processed.
> +
> + Throttling value set in register implies the approximate amount of delaying
> + the traffic between core and memory. The higher throttling value results in
^ remove 'The' ^ result
> + lower bandwidth. The max throttling value (MBA_MAX) supported can be
> + obtained through CPUID inside hypervisor. User can know it through
"Users can fetch the MBA_MAX value using the `psr-hwinfo` xl command."
> + `psr-hwinfo`.
> +
> +# Technical details
> +
> +MBA is a member of Intel PSR features, it shares the base PSR infrastructure
> +in Xen.
> +
> +## Hardware perspective
> +
> + MBA defines a range of MSRs to support specifying a delay value (Thrtl) per
> + COS, with details below.
> +
> + ```
> + +----------------------------+----------------+
> + | MSR (per socket) | Address |
> + +----------------------------+----------------+
> + | IA32_L2_QOS_Ext_BW_Thrtl_0 | 0xD50 |
> + +----------------------------+----------------+
> + | ... | ... |
> + +----------------------------+----------------+
> + | IA32_L2_QOS_Ext_BW_Thrtl_n | 0xD50+n |
> + +----------------------------+----------------+
> + ```
> +
> + When context switch happens, the COS ID of domain is written to per-hyper-
> + thread MSR `IA32_PQR_ASSOC`, and then hardware enforces bandwidth allocation
> + according to the throttling value stored in the Thrtl MSR register.
> +
> +## The relationship between MBA and CAT/CDP
> +
> + Generally speaking, MBA is completely independent of CAT/CDP, and any
> + combination may be applied at any time, e.g. enabling MBA with CAT
> + disabled.
> +
> + But it needs to be noticed that MBA shares COS infrastructure with CAT,
> + although MBA is enumerated by different CPUID leaf from CAT (which
> + indicates that the max COS of MBA may be different from CAT). In some
> + cases, a domain is permitted to have a COS that is beyond one (or more)
> + of PSR features but within the others. For instance, let's assume the max
> + COS of MBA is 8 but the max COS of L3 CAT is 16, when a domain is assigned
> + 9 as COS, the L3 CAT CBM associated to COS 9 would be enforced, but for MBA,
> + the HW works as default value is set since COS 9 is beyond the max COS (8)
> + of MBA.
> +
> +## Design Overview
> +
> +* Core COS/Thrtl association
> +
> + When enforcing Memory Bandwidth Allocation, all cores of domains have
> + the same default Thrtl MSR (COS0) which stores the same Thrtl (0). The
> + default Thrtl MSR is used only in hypervisor and is transparent to tool stack
> + and user.
> +
> + System administrators can change PSR allocation policy at runtime by
> + using the tool stack. Since MBA shares COS ID with CAT/CDP, a COS ID
> + corresponds to a 2-tuple, like [CBM, Thrtl] with only-CAT enabled, when CDP
> + is enabled, the COS ID corresponds to a 3-tuple, like [Code_CBM, Data_CBM,
> + Thrtl]. If neither CAT nor CDP is enabled, things are easier, since one COS
> + ID corresponds to one Thrtl.
I find the above paragraph a little bit difficult to parse, although
I'm not going to force you to re-write it.
> +
> +* VCPU schedule
> +
> + This part reuses CAT COS infrastructure.
> +
> +* Multi-sockets
> +
> + Different sockets may have different MBA ability (like max COS)
^ capabilities?
[...]
> +# Testing
> +
> +We can execute these commands to verify MBA on different HWs supporting them.
> +
> +For example:
> + 1. User can get the MBA hardware info through 'psr-hwinfo' command. From
> + result, user can know if this hardware works under linear mode or non-
> + linear mode, the max throttling value (MBA_MAX) and so on.
> +
> + root@:~$ xl psr-hwinfo --mba
> + Memory Bandwidth Allocation (MBA):
> + Socket ID : 0
> + Linear Mode : Enabled
> + Maximum COS : 7
> + Maximum Throttling Value: 90
> + Default Throttling Value: 0
> +
> + 2. Then, user can set a throttling value to a domain. For example, set '0xa',
> + i.e 10% delay.
> +
> + root@:~$ xl psr-mba-set 1 0xa
I would write this as 10 instead of 0xa, ie:
$ xl psr-mba-set 1 10
I think it's clearer because MBA is in linear mode, so the values
returned from xl will be in decimal base rather than hexadecimal.
Thanks, Roger.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v4 02/15] Rename PSR sysctl/domctl interfaces and xsm policy to make them be general
2017-09-23 9:48 ` [PATCH v4 02/15] Rename PSR sysctl/domctl interfaces and xsm policy to make them be general Yi Sun
@ 2017-09-25 16:15 ` Roger Pau Monné
2017-09-26 5:15 ` Yi Sun
2017-09-26 14:19 ` Wei Liu
2017-09-28 10:21 ` Jan Beulich
2 siblings, 1 reply; 64+ messages in thread
From: Roger Pau Monné @ 2017-09-25 16:15 UTC (permalink / raw)
To: Yi Sun
Cc: Wei Liu, Andrew Cooper, Ian Jackson, Jan Beulich, Chao Peng,
xen-devel, Daniel De Graaf
On Sat, Sep 23, 2017 at 09:48:11AM +0000, Yi Sun wrote:
> --- a/xen/arch/x86/domctl.c
> +++ b/xen/arch/x86/domctl.c
> @@ -1439,60 +1439,60 @@ long arch_do_domctl(
> }
> break;
>
> - case XEN_DOMCTL_psr_cat_op:
> - switch ( domctl->u.psr_cat_op.cmd )
> + case XEN_DOMCTL_psr_alloc:
> + switch ( domctl->u.psr_alloc.cmd )
> {
> uint32_t val32;
>
> - case XEN_DOMCTL_PSR_CAT_OP_SET_L3_CBM:
> - ret = psr_set_val(d, domctl->u.psr_cat_op.target,
> - domctl->u.psr_cat_op.data,
> + case XEN_DOMCTL_PSR_SET_L3_CBM:
> + ret = psr_set_val(d, domctl->u.psr_alloc.target,
> + domctl->u.psr_alloc.data,
> PSR_CBM_TYPE_L3);
> break;
>
> - case XEN_DOMCTL_PSR_CAT_OP_SET_L3_CODE:
> - ret = psr_set_val(d, domctl->u.psr_cat_op.target,
> - domctl->u.psr_cat_op.data,
> + case XEN_DOMCTL_PSR_SET_L3_CODE:
> + ret = psr_set_val(d, domctl->u.psr_alloc.target,
> + domctl->u.psr_alloc.data,
> PSR_CBM_TYPE_L3_CODE);
> break;
>
> - case XEN_DOMCTL_PSR_CAT_OP_SET_L3_DATA:
> - ret = psr_set_val(d, domctl->u.psr_cat_op.target,
> - domctl->u.psr_cat_op.data,
> + case XEN_DOMCTL_PSR_SET_L3_DATA:
> + ret = psr_set_val(d, domctl->u.psr_alloc.target,
> + domctl->u.psr_alloc.data,
> PSR_CBM_TYPE_L3_DATA);
> break;
>
> - case XEN_DOMCTL_PSR_CAT_OP_SET_L2_CBM:
> - ret = psr_set_val(d, domctl->u.psr_cat_op.target,
> - domctl->u.psr_cat_op.data,
> + case XEN_DOMCTL_PSR_SET_L2_CBM:
> + ret = psr_set_val(d, domctl->u.psr_alloc.target,
> + domctl->u.psr_alloc.data,
> PSR_CBM_TYPE_L2);
> break;
>
> - case XEN_DOMCTL_PSR_CAT_OP_GET_L3_CBM:
> - ret = psr_get_val(d, domctl->u.psr_cat_op.target,
> + case XEN_DOMCTL_PSR_GET_L3_CBM:
> + ret = psr_get_val(d, domctl->u.psr_alloc.target,
> &val32, PSR_CBM_TYPE_L3);
> - domctl->u.psr_cat_op.data = val32;
> + domctl->u.psr_alloc.data = val32;
> copyback = true;
> break;
>
> - case XEN_DOMCTL_PSR_CAT_OP_GET_L3_CODE:
> - ret = psr_get_val(d, domctl->u.psr_cat_op.target,
> + case XEN_DOMCTL_PSR_GET_L3_CODE:
> + ret = psr_get_val(d, domctl->u.psr_alloc.target,
> &val32, PSR_CBM_TYPE_L3_CODE);
> - domctl->u.psr_cat_op.data = val32;
> + domctl->u.psr_alloc.data = val32;
> copyback = true;
> break;
>
> - case XEN_DOMCTL_PSR_CAT_OP_GET_L3_DATA:
> - ret = psr_get_val(d, domctl->u.psr_cat_op.target,
> + case XEN_DOMCTL_PSR_GET_L3_DATA:
> + ret = psr_get_val(d, domctl->u.psr_alloc.target,
> &val32, PSR_CBM_TYPE_L3_DATA);
> - domctl->u.psr_cat_op.data = val32;
> + domctl->u.psr_alloc.data = val32;
> copyback = true;
> break;
>
> - case XEN_DOMCTL_PSR_CAT_OP_GET_L2_CBM:
> - ret = psr_get_val(d, domctl->u.psr_cat_op.target,
> + case XEN_DOMCTL_PSR_GET_L2_CBM:
> + ret = psr_get_val(d, domctl->u.psr_alloc.target,
> &val32, PSR_CBM_TYPE_L2);
> - domctl->u.psr_cat_op.data = val32;
> + domctl->u.psr_alloc.data = val32;
> copyback = true;
This:
ret = psr_get_val(...);
domctl->u.psr...;
copyback = true;
Construct is quite repetitive, maybe you could consider adding a local
macro to hide it?
Maybe then you could also hide the val32 declaration inside of it.
> break;
>
> diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
> index daa2aeb..c851511 100644
> --- a/xen/arch/x86/psr.c
> +++ b/xen/arch/x86/psr.c
> @@ -382,7 +382,7 @@ static bool l3_cdp_get_feat_info(const struct feat_node *feat,
> if ( !cat_get_feat_info(feat, data, array_len) )
> return false;
>
> - data[PSR_INFO_IDX_CAT_FLAG] |= XEN_SYSCTL_PSR_CAT_L3_CDP;
> + data[PSR_INFO_IDX_CAT_FLAG] |= XEN_SYSCTL_PSR_L3_CDP;
>
> return true;
> }
> diff --git a/xen/arch/x86/sysctl.c b/xen/arch/x86/sysctl.c
> index c3fdae8..e44d8ad 100644
> --- a/xen/arch/x86/sysctl.c
> +++ b/xen/arch/x86/sysctl.c
> @@ -171,45 +171,45 @@ long arch_do_sysctl(
>
> break;
>
> - case XEN_SYSCTL_psr_cat_op:
> - switch ( sysctl->u.psr_cat_op.cmd )
> + case XEN_SYSCTL_psr_alloc:
> + switch ( sysctl->u.psr_alloc.cmd )
> {
> uint32_t data[PSR_INFO_ARRAY_SIZE];
>
> - case XEN_SYSCTL_PSR_CAT_get_l3_info:
> + case XEN_SYSCTL_PSR_get_l3_info:
> {
> - ret = psr_get_info(sysctl->u.psr_cat_op.target,
> + ret = psr_get_info(sysctl->u.psr_alloc.target,
> PSR_CBM_TYPE_L3, data, ARRAY_SIZE(data));
> if ( ret )
> break;
>
> - sysctl->u.psr_cat_op.u.cat_info.cos_max =
> + sysctl->u.psr_alloc.u.cat_info.cos_max =
> data[PSR_INFO_IDX_COS_MAX];
> - sysctl->u.psr_cat_op.u.cat_info.cbm_len =
> + sysctl->u.psr_alloc.u.cat_info.cbm_len =
> data[PSR_INFO_IDX_CAT_CBM_LEN];
> - sysctl->u.psr_cat_op.u.cat_info.flags =
> + sysctl->u.psr_alloc.u.cat_info.flags =
> data[PSR_INFO_IDX_CAT_FLAG];
>
> - if ( __copy_field_to_guest(u_sysctl, sysctl, u.psr_cat_op) )
> + if ( __copy_field_to_guest(u_sysctl, sysctl, u.psr_alloc) )
> ret = -EFAULT;
> break;
> }
>
> - case XEN_SYSCTL_PSR_CAT_get_l2_info:
> + case XEN_SYSCTL_PSR_get_l2_info:
> {
> - ret = psr_get_info(sysctl->u.psr_cat_op.target,
> + ret = psr_get_info(sysctl->u.psr_alloc.target,
> PSR_CBM_TYPE_L2, data, ARRAY_SIZE(data));
> if ( ret )
> break;
>
> - sysctl->u.psr_cat_op.u.cat_info.cos_max =
> + sysctl->u.psr_alloc.u.cat_info.cos_max =
> data[PSR_INFO_IDX_COS_MAX];
> - sysctl->u.psr_cat_op.u.cat_info.cbm_len =
> + sysctl->u.psr_alloc.u.cat_info.cbm_len =
> data[PSR_INFO_IDX_CAT_CBM_LEN];
> - sysctl->u.psr_cat_op.u.cat_info.flags =
> + sysctl->u.psr_alloc.u.cat_info.flags =
> data[PSR_INFO_IDX_CAT_FLAG];
The above repetition is also unneeded AFAICT. Why not simplify the
switch to:
uint32_t data[PSR_INFO_ARRAY_SIZE];
enum cbm_type type;
switch ( sysctl->u.psr_alloc.cmd )
{
case XEN_SYSCTL_PSR_get_l3_info:
type = PSR_CBM_TYPE_L3;
break;
case XEN_SYSCTL_PSR_get_l2_info:
type = PSR_CBM_TYPE_L2;
break;
default:
return -EOPNOTSUPP;
}
ret = psr_get_info(sysctl->u.psr_alloc.target, type, data, ARRAY_SIZE(data));
if ( ret )
break;
sysctl->u.psr_alloc.u.cat_info.cos_max = data[PSR_INFO_IDX_COS_MAX];
...
It would prevent some of this code repetition IMHO.
Thanks, Roger.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v4 02/15] Rename PSR sysctl/domctl interfaces and xsm policy to make them be general
2017-09-25 16:15 ` Roger Pau Monné
@ 2017-09-26 5:15 ` Yi Sun
0 siblings, 0 replies; 64+ messages in thread
From: Yi Sun @ 2017-09-26 5:15 UTC (permalink / raw)
To: Roger Pau Monn�
Cc: Wei Liu, Andrew Cooper, Ian Jackson, Jan Beulich, Chao Peng,
xen-devel, Daniel De Graaf
On 17-09-25 17:15:21, Roger Pau Monn� wrote:
> On Sat, Sep 23, 2017 at 09:48:11AM +0000, Yi Sun wrote:
> > --- a/xen/arch/x86/domctl.c
> > +++ b/xen/arch/x86/domctl.c
[...]
> > - case XEN_DOMCTL_PSR_CAT_OP_GET_L3_DATA:
> > - ret = psr_get_val(d, domctl->u.psr_cat_op.target,
> > + case XEN_DOMCTL_PSR_GET_L3_DATA:
> > + ret = psr_get_val(d, domctl->u.psr_alloc.target,
> > &val32, PSR_CBM_TYPE_L3_DATA);
> > - domctl->u.psr_cat_op.data = val32;
> > + domctl->u.psr_alloc.data = val32;
> > copyback = true;
> > break;
> >
> > - case XEN_DOMCTL_PSR_CAT_OP_GET_L2_CBM:
> > - ret = psr_get_val(d, domctl->u.psr_cat_op.target,
> > + case XEN_DOMCTL_PSR_GET_L2_CBM:
> > + ret = psr_get_val(d, domctl->u.psr_alloc.target,
> > &val32, PSR_CBM_TYPE_L2);
> > - domctl->u.psr_cat_op.data = val32;
> > + domctl->u.psr_alloc.data = val32;
> > copyback = true;
>
> This:
>
> ret = psr_get_val(...);
> domctl->u.psr...;
> copyback = true;
>
> Construct is quite repetitive, maybe you could consider adding a local
> macro to hide it?
>
> Maybe then you could also hide the val32 declaration inside of it.
>
Thanks for the suggestion! Let me try.
[...]
> >
> > --- a/xen/arch/x86/sysctl.c
> > +++ b/xen/arch/x86/sysctl.c
> > @@ -171,45 +171,45 @@ long arch_do_sysctl(
> >
> > break;
> >
> > - case XEN_SYSCTL_psr_cat_op:
> > - switch ( sysctl->u.psr_cat_op.cmd )
> > + case XEN_SYSCTL_psr_alloc:
> > + switch ( sysctl->u.psr_alloc.cmd )
> > {
> > uint32_t data[PSR_INFO_ARRAY_SIZE];
> >
> > - case XEN_SYSCTL_PSR_CAT_get_l3_info:
> > + case XEN_SYSCTL_PSR_get_l3_info:
> > {
> > - ret = psr_get_info(sysctl->u.psr_cat_op.target,
> > + ret = psr_get_info(sysctl->u.psr_alloc.target,
> > PSR_CBM_TYPE_L3, data, ARRAY_SIZE(data));
> > if ( ret )
> > break;
> >
> > - sysctl->u.psr_cat_op.u.cat_info.cos_max =
> > + sysctl->u.psr_alloc.u.cat_info.cos_max =
> > data[PSR_INFO_IDX_COS_MAX];
> > - sysctl->u.psr_cat_op.u.cat_info.cbm_len =
> > + sysctl->u.psr_alloc.u.cat_info.cbm_len =
> > data[PSR_INFO_IDX_CAT_CBM_LEN];
> > - sysctl->u.psr_cat_op.u.cat_info.flags =
> > + sysctl->u.psr_alloc.u.cat_info.flags =
> > data[PSR_INFO_IDX_CAT_FLAG];
> >
> > - if ( __copy_field_to_guest(u_sysctl, sysctl, u.psr_cat_op) )
> > + if ( __copy_field_to_guest(u_sysctl, sysctl, u.psr_alloc) )
> > ret = -EFAULT;
> > break;
> > }
> >
> > - case XEN_SYSCTL_PSR_CAT_get_l2_info:
> > + case XEN_SYSCTL_PSR_get_l2_info:
> > {
> > - ret = psr_get_info(sysctl->u.psr_cat_op.target,
> > + ret = psr_get_info(sysctl->u.psr_alloc.target,
> > PSR_CBM_TYPE_L2, data, ARRAY_SIZE(data));
> > if ( ret )
> > break;
> >
> > - sysctl->u.psr_cat_op.u.cat_info.cos_max =
> > + sysctl->u.psr_alloc.u.cat_info.cos_max =
> > data[PSR_INFO_IDX_COS_MAX];
> > - sysctl->u.psr_cat_op.u.cat_info.cbm_len =
> > + sysctl->u.psr_alloc.u.cat_info.cbm_len =
> > data[PSR_INFO_IDX_CAT_CBM_LEN];
> > - sysctl->u.psr_cat_op.u.cat_info.flags =
> > + sysctl->u.psr_alloc.u.cat_info.flags =
> > data[PSR_INFO_IDX_CAT_FLAG];
>
> The above repetition is also unneeded AFAICT. Why not simplify the
> switch to:
>
> uint32_t data[PSR_INFO_ARRAY_SIZE];
> enum cbm_type type;
>
> switch ( sysctl->u.psr_alloc.cmd )
> {
> case XEN_SYSCTL_PSR_get_l3_info:
> type = PSR_CBM_TYPE_L3;
> break;
> case XEN_SYSCTL_PSR_get_l2_info:
> type = PSR_CBM_TYPE_L2;
> break;
> default:
> return -EOPNOTSUPP;
> }
>
> ret = psr_get_info(sysctl->u.psr_alloc.target, type, data, ARRAY_SIZE(data));
> if ( ret )
> break;
>
> sysctl->u.psr_alloc.u.cat_info.cos_max = data[PSR_INFO_IDX_COS_MAX];
> ...
>
> It would prevent some of this code repetition IMHO.
>
Thank you! But this way seems only good to CAT features but could not cover MBA.
Although I can assign "type = PSR_TYPE_MBA_THRTL" for MBA, the assignment of
'sysctl->u.psr_alloc.u.mba_info' and 'sysctl->u.psr_alloc.u.cat_info' cannot be
converged. We have to check the type to decide to assign value to which 'info'.
> Thanks, Roger.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v4 04/15] x86: implement data structure and CPU init flow for MBA
2017-09-23 9:48 ` [PATCH v4 04/15] x86: implement data structure and CPU init flow for MBA Yi Sun
@ 2017-09-26 8:38 ` Roger Pau Monné
2017-09-28 10:27 ` Jan Beulich
2017-09-28 11:00 ` Jan Beulich
1 sibling, 1 reply; 64+ messages in thread
From: Roger Pau Monné @ 2017-09-26 8:38 UTC (permalink / raw)
To: Yi Sun; +Cc: xen-devel, Wei Liu, Chao Peng, Jan Beulich, Andrew Cooper
On Sat, Sep 23, 2017 at 09:48:13AM +0000, Yi Sun wrote:
> This patch implements main data structures of MBA.
>
> Like CAT features, MBA HW info has cos_max which means the max thrtl
> register number, and thrtl_max which means the max throttle value
> (delay value). It also has a flag to represent if the throttle
> value is linear or not.
>
> One thrtl register of MBA stores a throttle value for one or more
> domains. The throttle value means the delay between L2 cache and next
> cache level.
>
> This patch also implements init flow for MBA and register stub
> callback functions.
>
> Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
With two nits below.
> @@ -332,20 +355,58 @@ static int cat_init_feature(const struct cpuid_leaf *regs,
> }
>
> default:
> - return -ENOENT;
> + return false;
> }
>
> /* Add this feature into array. */
> info->features[type] = feat;
>
> if ( !opt_cpu_info )
> - return 0;
> + return true;
>
> printk(XENLOG_INFO "%s: enabled on socket %u, cos_max:%u, cbm_len:%u\n",
> cat_feat_name[type], cpu_to_socket(smp_processor_id()),
> - feat->cos_max, feat->cbm_len);
> + feat->cos_max, feat->cat.cbm_len);
I would rather do:
if ( opt_cpu_info )
printk(...);
return true;
So that the function has a single return path for the success case.
>
> - return 0;
> + return true;
> +}
> +
> +static bool mba_init_feature(const struct cpuid_leaf *regs,
> + struct feat_node *feat,
> + struct psr_socket_info *info,
> + enum psr_feat_type type)
> +{
> + /* No valid value so do not enable feature. */
> + if ( !regs->a || !regs->d || type != FEAT_TYPE_MBA )
> + return false;
> +
> + feat->cos_max = min(opt_cos_max, regs->d & CAT_COS_MAX_MASK);
> + if ( feat->cos_max < 1 )
> + return false;
> +
> + feat->mba.thrtl_max = (regs->a & MBA_THRTL_MAX_MASK) + 1;
> +
> + if ( regs->c & MBA_LINEAR_MASK )
> + {
> + feat->mba.linear = true;
> +
> + if ( feat->mba.thrtl_max >= 100 )
> + return false;
> + }
> +
> + wrmsrl(MSR_IA32_PSR_MBA_MASK(0), 0);
> +
> + /* Add this feature into array. */
> + info->features[type] = feat;
> +
> + if ( !opt_cpu_info )
> + return true;
> +
> + printk(XENLOG_INFO "MBA: enabled on socket %u, cos_max:%u, thrtl_max:%u, linear:%u.\n",
^ newline.
Thanks, Roger.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v4 05/15] x86: implement get hw info flow for MBA
2017-09-23 9:48 ` [PATCH v4 05/15] x86: implement get hw info " Yi Sun
@ 2017-09-26 8:46 ` Roger Pau Monné
2017-09-26 11:49 ` Jan Beulich
0 siblings, 1 reply; 64+ messages in thread
From: Roger Pau Monné @ 2017-09-26 8:46 UTC (permalink / raw)
To: Yi Sun; +Cc: xen-devel, Wei Liu, Chao Peng, Jan Beulich, Andrew Cooper
On Sat, Sep 23, 2017 at 09:48:14AM +0000, Yi Sun wrote:
> This patch implements get HW info flow for MBA including its callback
> function and sysctl interface.
>
> Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
> ---
> CC: Jan Beulich <jbeulich@suse.com>
> CC: Andrew Cooper <andrew.cooper3@citrix.com>
> CC: Wei Liu <wei.liu2@citrix.com>
> CC: Roger Pau Monné <roger.pau@citrix.com>
> CC: Chao Peng <chao.p.peng@linux.intel.com>
>
> v4:
> - remove 'ALLOC_' from macro names.
> (suggested by Roger Pau Monné)
> - initialize 'data[PSR_INFO_ARRAY_SIZE]' to 0 to prevent to leak stack data.
> (suggested by Roger Pau Monné)
> v3:
> - replace 'PSR_VAL_TYPE_MBA' to 'PSR_TYPE_MBA_THRTL'.
> (suggested by Roger Pau Monné)
> v2:
> - use 'XEN_SYSCTL_PSR_MBA_LINEAR' to set MBA feature HW info.
> (suggested by Chao Peng)
> v1:
> - sort 'PSR_INFO_IDX_' macros as feature.
> (suggested by Chao Peng)
> - rename 'PSR_INFO_IDX_MBA_LINEAR' to 'PSR_INFO_IDX_MBA_FLAG'.
> - rename 'linear' in 'struct mba_info' to 'flags' for future extension.
> (suggested by Chao Peng)
> ---
> xen/arch/x86/psr.c | 15 ++++++++++++++-
> xen/arch/x86/sysctl.c | 21 ++++++++++++++++++++-
> xen/include/asm-x86/psr.h | 2 ++
> xen/include/public/sysctl.h | 8 ++++++++
> 4 files changed, 44 insertions(+), 2 deletions(-)
>
> diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
> index e2f8fd0..1d0a215 100644
> --- a/xen/arch/x86/psr.c
> +++ b/xen/arch/x86/psr.c
> @@ -263,6 +263,10 @@ static enum psr_feat_type psr_type_to_feat_type(enum psr_type type)
> feat_type = FEAT_TYPE_L2_CAT;
> break;
>
> + case PSR_TYPE_MBA_THRTL:
> + feat_type = FEAT_TYPE_MBA;
> + break;
> +
> default:
> ASSERT_UNREACHABLE();
> }
> @@ -486,7 +490,16 @@ static const struct feat_props l2_cat_props = {
> static bool mba_get_feat_info(const struct feat_node *feat,
> uint32_t data[], unsigned int array_len)
> {
> - return false;
> + if ( array_len != PSR_INFO_ARRAY_SIZE )
Should this be an ASSERT maybe? I don't see how array_len can be !=
PSR_INFO_ARRAY_SIZE.
> + return false;
> +
> + data[PSR_INFO_IDX_COS_MAX] = feat->cos_max;
> + data[PSR_INFO_IDX_MBA_THRTL_MAX] = feat->mba.thrtl_max;
> +
> + if ( feat->mba.linear )
> + data[PSR_INFO_IDX_MBA_FLAG] |= XEN_SYSCTL_PSR_MBA_LINEAR;
> +
> + return true;
> }
>
> static void mba_write_msr(unsigned int cos, uint32_t val,
> diff --git a/xen/arch/x86/sysctl.c b/xen/arch/x86/sysctl.c
> index 9dee163..b2f8efa 100644
> --- a/xen/arch/x86/sysctl.c
> +++ b/xen/arch/x86/sysctl.c
> @@ -174,7 +174,7 @@ long arch_do_sysctl(
> case XEN_SYSCTL_psr_alloc:
> switch ( sysctl->u.psr_alloc.cmd )
> {
> - uint32_t data[PSR_INFO_ARRAY_SIZE];
> + uint32_t data[PSR_INFO_ARRAY_SIZE] = {0};
I would rather do "{ 0 };" or "{ };" (note the spaces around the 0).
Thanks, Roger.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v4 07/15] x86: implement set value flow for MBA
2017-09-23 9:48 ` [PATCH v4 07/15] x86: implement set value flow " Yi Sun
@ 2017-09-26 9:39 ` Roger Pau Monné
2017-09-28 2:39 ` Yi Sun
2017-09-28 11:36 ` Jan Beulich
1 sibling, 1 reply; 64+ messages in thread
From: Roger Pau Monné @ 2017-09-26 9:39 UTC (permalink / raw)
To: Yi Sun; +Cc: xen-devel, Wei Liu, Chao Peng, Jan Beulich, Andrew Cooper
On Sat, Sep 23, 2017 at 09:48:16AM +0000, Yi Sun wrote:
> @@ -274,29 +277,6 @@ static enum psr_feat_type psr_type_to_feat_type(enum psr_type type)
> return feat_type;
> }
>
> -static bool psr_check_cbm(unsigned int cbm_len, unsigned long cbm)
> -{
> - unsigned int first_bit, zero_bit;
> -
> - /* Set bits should only in the range of [0, cbm_len]. */
> - if ( cbm & (~0ul << cbm_len) )
> - return false;
> -
> - /* At least one bit need to be set. */
> - if ( cbm == 0 )
> - return false;
> -
> - first_bit = find_first_bit(&cbm, cbm_len);
> - zero_bit = find_next_zero_bit(&cbm, cbm_len, first_bit);
> -
> - /* Set bits should be contiguous. */
> - if ( zero_bit < cbm_len &&
> - find_next_bit(&cbm, cbm_len, zero_bit) < cbm_len )
> - return false;
> -
> - return true;
> -}
> -
> /* Implementation of allocation features' functions. */
> static bool cat_init_feature(const struct cpuid_leaf *regs,
> struct feat_node *feat,
> @@ -426,11 +406,36 @@ static bool cat_get_feat_info(const struct feat_node *feat,
> return true;
> }
>
> +static bool cat_check_cbm(const struct feat_node *feat, unsigned long cbm)
> +{
> + unsigned int first_bit, zero_bit;
> + unsigned int cbm_len = feat->cat.cbm_len;
> +
> + /*
> + * Set bits should only in the range of [0, cbm_len].
> + * And, at least one bit need to be set.
> + */
> + if ( cbm & (~0ul << cbm_len) || cbm == 0 )
> + return false;
> +
> + first_bit = find_first_bit(&cbm, cbm_len);
> + zero_bit = find_next_zero_bit(&cbm, cbm_len, first_bit);
> +
> + /* Set bits should be contiguous. */
> + if ( zero_bit < cbm_len &&
> + find_next_bit(&cbm, cbm_len, zero_bit) < cbm_len )
> + return false;
> +
> + return true;
> +}
Why do you need to move the code apart from renaming it?
> @@ -1210,25 +1237,39 @@ static unsigned int get_socket_cpu(unsigned int socket)
> struct cos_write_info
> {
> unsigned int cos;
> - struct feat_node *feature;
> + struct feat_node **features;
> const uint32_t *val;
> - const struct feat_props *props;
> + unsigned int array_len;
> + const struct feat_props **props;
Why do you need props here, from the usage below it's just pointing
to feat_props, which is already available in this context.
> };
>
> static void do_write_psr_msrs(void *data)
> {
> const struct cos_write_info *info = data;
> - struct feat_node *feat = info->feature;
> - const struct feat_props *props = info->props;
> - unsigned int i, cos = info->cos, cos_num = props->cos_num;
> + unsigned int i, index = 0, cos = info->cos;
> + const uint32_t *val_array = info->val;
>
> - for ( i = 0; i < cos_num; i++ )
> + for ( i = 0; i < ARRAY_SIZE(feat_props); i++ )
> {
> - if ( feat->cos_reg_val[cos * cos_num + i] != info->val[i] )
> + struct feat_node *feat = info->features[i];
> + const struct feat_props *props = info->props[i];
If you use ARRAY_SIZE(feat_props), the above should be feat_props[i].
Also I'm worried about the size of the props array, isn't there a
possibility that the props array is smaller than the feature array?
Thanks, Roger.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v4 08/15] tools: create general interfaces to support psr allocation features
2017-09-23 9:48 ` [PATCH v4 08/15] tools: create general interfaces to support psr allocation features Yi Sun
@ 2017-09-26 10:11 ` Roger Pau Monné
0 siblings, 0 replies; 64+ messages in thread
From: Roger Pau Monné @ 2017-09-26 10:11 UTC (permalink / raw)
To: Yi Sun; +Cc: xen-devel, Wei Liu, Ian Jackson, Chao Peng
On Sat, Sep 23, 2017 at 09:48:17AM +0000, Yi Sun wrote:
> This patch creates general interfaces in libxl to support all psr
> allocation features.
>
> Add 'LIBXL_HAVE_PSR_GENERIC' to indicate interface change.
>
> Please note, the functionality cannot work until later patches
> are applied.
>
> Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
> ---
> CC: Wei Liu <wei.liu2@citrix.com>
> CC: Ian Jackson <ian.jackson@eu.citrix.com>
> CC: Roger Pau Monné <roger.pau@citrix.com>
> CC: Chao Peng <chao.p.peng@linux.intel.com>
>
> v4:
> - add description for LIBXL_HAVE_PSR_GENERIC to mention newly added
> public functions.
> (suggested by Roger Pau Monné)
> v3:
> - change 'LIBXL_HAVE_PSR_MBA' to 'LIBXL_HAVE_PSR_GENERIC'.
> (suggested by Roger Pau Monné)
> - 'typedef enum libxl_psr_cbm_type libxl_psr_type;' in libxl.h.
> (suggested by Roger Pau Monné and Wei Liu)
> - change 'libxl_psr_cbm_type' to 'libxl_psr_type' in newly defined
> interfaces.
> (suggested by Roger Pau Monné)
> v2:
> - remove '_INFO' in 'libxl_psr_feat_type' and make corresponding
> changes in 'libxl_psr_hw_info'.
> (suggested by Chao Peng)
> ---
> tools/libxl/libxl.h | 37 +++++++++++++++++++++++++++++++++++++
> tools/libxl/libxl_psr.c | 25 +++++++++++++++++++++++++
> tools/libxl/libxl_types.idl | 22 ++++++++++++++++++++++
> 3 files changed, 84 insertions(+)
>
> diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h
> index 91408b4..569c331 100644
> --- a/tools/libxl/libxl.h
> +++ b/tools/libxl/libxl.h
> @@ -936,6 +936,17 @@ void libxl_mac_copy(libxl_ctx *ctx, libxl_mac *dst, const libxl_mac *src);
> #define LIBXL_HAVE_PSR_L2_CAT 1
>
> /*
> + * LIBXL_HAVE_PSR_GENERIC
> + *
> + * If this is defined, the Memory Bandwidth Allocation feature is supported.
> + * The following public functions are available:
> + * libxl_psr_{set/get}_val
> + * libxl_psr_get_hw_info
> + * libxl_psr_hw_info_list_free
> + */
> +#define LIBXL_HAVE_PSR_GENERIC 1
> +
> +/*
> * LIBXL_HAVE_MCA_CAPS
> *
> * If this is defined, setting MCA capabilities for HVM domain is supported.
> @@ -2228,6 +2239,32 @@ int libxl_psr_cat_get_info(libxl_ctx *ctx, libxl_psr_cat_info **info,
> int libxl_psr_cat_get_l3_info(libxl_ctx *ctx, libxl_psr_cat_info **info,
> int *nr);
> void libxl_psr_cat_info_list_free(libxl_psr_cat_info *list, int nr);
> +
> +typedef enum libxl_psr_cbm_type libxl_psr_type;
> +
> +/*
> + * Function to set a domain's value. It operates on a single or multiple
> + * target(s) defined in 'target_map'. 'target_map' specifies all the sockets
> + * to be operated on.
> + */
> +int libxl_psr_set_val(libxl_ctx *ctx, uint32_t domid,
> + libxl_psr_type type, libxl_bitmap *target_map,
> + uint64_t val);
> +/*
> + * Function to get a domain's cbm. It operates on a single 'target'.
> + * 'target' specifies which socket to be operated on.
> + */
> +int libxl_psr_get_val(libxl_ctx *ctx, uint32_t domid,
> + libxl_psr_type type, unsigned int target,
> + uint64_t *val);
> +/*
> + * On success, the function returns an array of elements in 'info',
> + * and the length in 'nr'.
> + */
> +int libxl_psr_get_hw_info(libxl_ctx *ctx, libxl_psr_hw_info **info,
> + unsigned int *nr, libxl_psr_feat_type type,
> + unsigned int lvl);
I would place the returned parameters at the end of the list, like you
do in libxl_psr_get_val.
> +void libxl_psr_hw_info_list_free(libxl_psr_hw_info *list, unsigned int nr);
> #endif
>
> /* misc */
> diff --git a/tools/libxl/libxl_psr.c b/tools/libxl/libxl_psr.c
> index 197505a..4a6978e 100644
> --- a/tools/libxl/libxl_psr.c
> +++ b/tools/libxl/libxl_psr.c
> @@ -428,6 +428,31 @@ void libxl_psr_cat_info_list_free(libxl_psr_cat_info *list, int nr)
> free(list);
> }
>
> +int libxl_psr_set_val(libxl_ctx *ctx, uint32_t domid,
> + libxl_psr_type type, libxl_bitmap *target_map,
> + uint64_t val)
> +{
> + return ERROR_FAIL;
> +}
> +
> +int libxl_psr_get_val(libxl_ctx *ctx, uint32_t domid,
> + libxl_psr_type type, unsigned int target,
> + uint64_t *val)
> +{
> + return ERROR_FAIL;
> +}
> +
> +int libxl_psr_get_hw_info(libxl_ctx *ctx, libxl_psr_hw_info **info,
> + unsigned int *nr, libxl_psr_feat_type type,
> + unsigned int lvl)
> +{
> + return ERROR_FAIL;
> +}
> +
> +void libxl_psr_hw_info_list_free(libxl_psr_hw_info *list, unsigned int nr)
> +{
> +}
> +
> /*
> * Local variables:
> * mode: C
> diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl
> index 173d70a..cfe8367 100644
> --- a/tools/libxl/libxl_types.idl
> +++ b/tools/libxl/libxl_types.idl
> @@ -977,6 +977,7 @@ libxl_psr_cbm_type = Enumeration("psr_cbm_type", [
> (2, "L3_CBM_CODE"),
> (3, "L3_CBM_DATA"),
> (4, "L2_CBM"),
> + (5, "MBA_THRTL"),
> ])
>
> libxl_psr_cat_info = Struct("psr_cat_info", [
> @@ -985,3 +986,24 @@ libxl_psr_cat_info = Struct("psr_cat_info", [
> ("cbm_len", uint32),
> ("cdp_enabled", bool),
> ])
> +
> +libxl_psr_feat_type = Enumeration("psr_feat_type", [
> + (1, "CAT"),
> + (2, "MBA"),
> + ])
So we have a libxl_psr_type and a libxl_psr_feat_type. This seems
quite unnecessary, but I guess now it's too late to fix this anyway.
Also I see that one is needed for libxl_psr_{set/get}_val while the
other is used for libxl_psr_get_hw_info.
Thanks, Roger.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v4 09/15] tools: implement the new libxc get hw info interface
2017-09-23 9:48 ` [PATCH v4 09/15] tools: implement the new libxc get hw info interface Yi Sun
@ 2017-09-26 10:22 ` Roger Pau Monné
0 siblings, 0 replies; 64+ messages in thread
From: Roger Pau Monné @ 2017-09-26 10:22 UTC (permalink / raw)
To: Yi Sun; +Cc: xen-devel, Wei Liu, Ian Jackson, Chao Peng
On Sat, Sep 23, 2017 at 09:48:18AM +0000, Yi Sun wrote:
> +enum xc_psr_feat_type {
> + XC_PSR_CAT_L3,
> + XC_PSR_CAT_L2,
> + XC_PSR_MBA,
> +};
> +typedef enum xc_psr_feat_type xc_psr_feat_type;
> +
> +struct xc_psr_hw_info {
> + union {
> + struct {
> + uint32_t cos_max;
> + uint32_t cbm_len;
> + bool cdp_enabled;
> + } cat;
> +
> + struct {
> + uint32_t cos_max;
> + uint32_t thrtl_max;
> + bool linear;
> + } mba;
> + } u;
> +};
Why not:
union xc_psr_hw_info {
struct {
uint32_t cos_max;
uint32_t cbm_len;
bool cdp_enabled;
} cat;
struct {
uint32_t cos_max;
uint32_t thrtl_max;
bool linear;
} mba;
};
> +typedef struct xc_psr_hw_info xc_psr_hw_info;
> +
> int xc_psr_cmt_attach(xc_interface *xch, uint32_t domid);
> int xc_psr_cmt_detach(xc_interface *xch, uint32_t domid);
> int xc_psr_cmt_get_domain_rmid(xc_interface *xch, uint32_t domid,
> @@ -2480,9 +2504,8 @@ int xc_psr_cat_set_domain_data(xc_interface *xch, uint32_t domid,
> int xc_psr_cat_get_domain_data(xc_interface *xch, uint32_t domid,
> xc_psr_cat_type type, uint32_t target,
> uint64_t *data);
> -int xc_psr_cat_get_info(xc_interface *xch, uint32_t socket, unsigned int lvl,
> - uint32_t *cos_max, uint32_t *cbm_len,
> - bool *cdp_enabled);
> +int xc_psr_get_hw_info(xc_interface *xch, uint32_t socket,
> + xc_psr_feat_type type, xc_psr_hw_info *hw_info);
>
> int xc_get_cpu_levelling_caps(xc_interface *xch, uint32_t *caps);
> int xc_get_cpu_featureset(xc_interface *xch, uint32_t index,
> diff --git a/tools/libxc/xc_psr.c b/tools/libxc/xc_psr.c
> index 623e26b..f2b5ee6 100644
> --- a/tools/libxc/xc_psr.c
> +++ b/tools/libxc/xc_psr.c
> @@ -323,37 +323,57 @@ int xc_psr_cat_get_domain_data(xc_interface *xch, uint32_t domid,
> return rc;
> }
>
> -int xc_psr_cat_get_info(xc_interface *xch, uint32_t socket, unsigned int lvl,
> - uint32_t *cos_max, uint32_t *cbm_len, bool *cdp_enabled)
> +int xc_psr_get_hw_info(xc_interface *xch, uint32_t socket,
> + xc_psr_feat_type type, xc_psr_hw_info *hw_info)
> {
> int rc = -1;
> DECLARE_SYSCTL;
>
> + if ( !hw_info )
> + {
> + errno = EINVAL;
> + return rc;
> + }
> +
> sysctl.cmd = XEN_SYSCTL_psr_alloc;
> sysctl.u.psr_alloc.target = socket;
>
> - switch ( lvl )
> + switch ( type )
> {
> - case 2:
> + case XC_PSR_CAT_L2:
> sysctl.u.psr_alloc.cmd = XEN_SYSCTL_PSR_get_l2_info;
> rc = xc_sysctl(xch, &sysctl);
> - if ( !rc )
> - {
> - *cos_max = sysctl.u.psr_alloc.u.cat_info.cos_max;
> - *cbm_len = sysctl.u.psr_alloc.u.cat_info.cbm_len;
> - *cdp_enabled = false;
> - }
> + if ( rc )
> + break;
> +
> + hw_info->u.cat.cos_max = sysctl.u.psr_alloc.u.cat_info.cos_max;
> + hw_info->u.cat.cbm_len = sysctl.u.psr_alloc.u.cat_info.cbm_len;
> + hw_info->u.cat.cdp_enabled = false;
> +
> break;
> - case 3:
> + case XC_PSR_CAT_L3:
> sysctl.u.psr_alloc.cmd = XEN_SYSCTL_PSR_get_l3_info;
> rc = xc_sysctl(xch, &sysctl);
> - if ( !rc )
> - {
> - *cos_max = sysctl.u.psr_alloc.u.cat_info.cos_max;
> - *cbm_len = sysctl.u.psr_alloc.u.cat_info.cbm_len;
> - *cdp_enabled = sysctl.u.psr_alloc.u.cat_info.flags &
> - XEN_SYSCTL_PSR_L3_CDP;
> - }
> + if ( rc )
> + break;
> +
> + hw_info->u.cat.cos_max = sysctl.u.psr_alloc.u.cat_info.cos_max;
> + hw_info->u.cat.cbm_len = sysctl.u.psr_alloc.u.cat_info.cbm_len;
> + hw_info->u.cat.cdp_enabled = sysctl.u.psr_alloc.u.cat_info.flags &
> + XEN_SYSCTL_PSR_L3_CDP;
The two cases (XC_PSR_CAT_L2 and XC_PSR_CAT_L3) can be easily joined
into a single one IMHO.
Thanks, Roger.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v4 10/15] tools: implement the new libxl get hw info interface
2017-09-23 9:48 ` [PATCH v4 10/15] tools: implement the new libxl " Yi Sun
@ 2017-09-26 10:54 ` Roger Pau Monné
0 siblings, 0 replies; 64+ messages in thread
From: Roger Pau Monné @ 2017-09-26 10:54 UTC (permalink / raw)
To: Yi Sun; +Cc: xen-devel, Wei Liu, Ian Jackson, Chao Peng
On Sat, Sep 23, 2017 at 09:48:19AM +0000, Yi Sun wrote:
> This patch implements the new libxl get hw info interface,
> 'libxl_psr_get_hw_info', which is suitable to all psr allocation
> features. It also implements corresponding list free function,
> 'libxl_psr_hw_info_list_free' and makes 'libxl_psr_cat_get_info' call
> 'libxl_psr_get_hw_info' to avoid redundant code in libxl_psr.c.
>
> Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
With one nit:
> int libxl_psr_get_hw_info(libxl_ctx *ctx, libxl_psr_hw_info **info,
> unsigned int *nr, libxl_psr_feat_type type,
> unsigned int lvl)
> {
> - return ERROR_FAIL;
> + GC_INIT(ctx);
> + int rc, nr_sockets;
> + unsigned int i = 0, socketid;
> + libxl_bitmap socketmap;
> + libxl_psr_hw_info *ptr;
> + xc_psr_feat_type xc_type;
> + xc_psr_hw_info hw_info;
> +
> + libxl_bitmap_init(&socketmap);
> +
> + xc_type = libxl__feat_type_to_libxc_feat_type(type, lvl);
> +
> + rc = libxl__count_physical_sockets(gc, &nr_sockets);
> + if (rc) {
> + LOG(ERROR, "failed to get system socket count");
> + goto out;
> + }
> +
> + libxl_socket_bitmap_alloc(ctx, &socketmap, nr_sockets);
> + rc = libxl_get_online_socketmap(ctx, &socketmap);
> + if (rc < 0) {
"if (rc)" like above.
Thanks, Roger.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v4 11/15] tools: implement the new xl get hw info interface
2017-09-23 9:48 ` [PATCH v4 11/15] tools: implement the new xl " Yi Sun
@ 2017-09-26 11:19 ` Roger Pau Monné
0 siblings, 0 replies; 64+ messages in thread
From: Roger Pau Monné @ 2017-09-26 11:19 UTC (permalink / raw)
To: Yi Sun; +Cc: xen-devel, Wei Liu, Ian Jackson, Chao Peng
On Sat, Sep 23, 2017 at 09:48:20AM +0000, Yi Sun wrote:
> This patch implements a new xl get HW info interface. A new argument
> is added for psr-hwinfo command to get and show MBA HW info.
>
> Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
Thanks, Roger.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v4 12/15] tools: rename 'xc_psr_cat_type' to 'xc_psr_type'
2017-09-23 9:48 ` [PATCH v4 12/15] tools: rename 'xc_psr_cat_type' to 'xc_psr_type' Yi Sun
@ 2017-09-26 11:21 ` Roger Pau Monné
0 siblings, 0 replies; 64+ messages in thread
From: Roger Pau Monné @ 2017-09-26 11:21 UTC (permalink / raw)
To: Yi Sun; +Cc: xen-devel, Wei Liu, Ian Jackson, Chao Peng
On Sat, Sep 23, 2017 at 09:48:21AM +0000, Yi Sun wrote:
> This patch renames 'xc_psr_cat_type' to 'xc_psr_type' so that
> the structure name is common for all allocation features.
>
> Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
> Acked-by: Wei Liu <wei.liu2@citrix.com>
> Reviewed-by: Chao Peng <chao.p.peng@linux.intel.com>
Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
Just a small nit:
> int libxl_psr_cat_set_cbm(libxl_ctx *ctx, uint32_t domid,
> @@ -325,12 +325,11 @@ int libxl_psr_cat_set_cbm(libxl_ctx *ctx, uint32_t domid,
> }
>
> libxl_for_each_set_bit(socketid, *target_map) {
> - xc_psr_cat_type xc_type;
> + xc_psr_type xc_type = libxl__psr_cbm_type_to_libxc_psr_type(type);;
Double ';' at the end of line.
Roger.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v4 13/15] tools: implement new generic get value interface and MBA get value command
2017-09-23 9:48 ` [PATCH v4 13/15] tools: implement new generic get value interface and MBA get value command Yi Sun
@ 2017-09-26 11:34 ` Roger Pau Monné
0 siblings, 0 replies; 64+ messages in thread
From: Roger Pau Monné @ 2017-09-26 11:34 UTC (permalink / raw)
To: Yi Sun; +Cc: xen-devel, Wei Liu, Ian Jackson, Chao Peng
On Sat, Sep 23, 2017 at 09:48:22AM +0000, Yi Sun wrote:
> This patch implements generic get value interfaces in libxc and libxl.
> It also refactors the get value flow in xl to make it be suitable for all
> allocation features. Based on that, a new MBA get value command is added in xl.
>
> Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
> Acked-by: Wei Liu <wei.liu2@citrix.com>
> ---
> CC: Wei Liu <wei.liu2@citrix.com>
> CC: Ian Jackson <ian.jackson@eu.citrix.com>
> CC: Roger Pau Monné <roger.pau@citrix.com>
> CC: Chao Peng <chao.p.peng@linux.intel.com>
>
> v4:
> - use designated initializers for 'feat_name[]'.
> (suggested by Roger Pau Monné)
> - use LOG in 'libxl__psr_alloc_log_err_msg'.
> (suggested by Roger Pau Monné)
> v3:
> - replace 'libxl_psr_cbm_type' to 'libxl_psr_type' in newly defined
> interfaces.
> (suggested by Roger Pau Monné)
> v2:
> - change 'CAT_INFO'/'MBA_INFO' to 'CAT'/'MBA'. The related structure names
> are changed too.
> (suggested by Chao Peng)
> ---
> tools/libxc/include/xenctrl.h | 7 +-
> tools/libxc/xc_psr.c | 9 +-
> tools/libxl/libxl_psr.c | 58 ++++++++-----
> tools/xl/xl.h | 1 +
> tools/xl/xl_cmdtable.c | 5 ++
> tools/xl/xl_psr.c | 185 ++++++++++++++++++++++++++++++------------
> 6 files changed, 183 insertions(+), 82 deletions(-)
>
> diff --git a/tools/libxc/include/xenctrl.h b/tools/libxc/include/xenctrl.h
> index 1cc4d7e..745b2a5 100644
> --- a/tools/libxc/include/xenctrl.h
> +++ b/tools/libxc/include/xenctrl.h
> @@ -2456,6 +2456,7 @@ enum xc_psr_type {
> XC_PSR_CAT_L3_CBM_CODE = 2,
> XC_PSR_CAT_L3_CBM_DATA = 3,
> XC_PSR_CAT_L2_CBM = 4,
> + XC_PSR_MBA_THRTL = 5,
> };
> typedef enum xc_psr_type xc_psr_type;
>
> @@ -2501,9 +2502,9 @@ int xc_psr_cmt_enabled(xc_interface *xch);
> int xc_psr_cat_set_domain_data(xc_interface *xch, uint32_t domid,
> xc_psr_type type, uint32_t target,
> uint64_t data);
> -int xc_psr_cat_get_domain_data(xc_interface *xch, uint32_t domid,
> - xc_psr_type type, uint32_t target,
> - uint64_t *data);
> +int xc_psr_get_domain_data(xc_interface *xch, uint32_t domid,
> + xc_psr_type type, uint32_t target,
> + uint64_t *data);
> int xc_psr_get_hw_info(xc_interface *xch, uint32_t socket,
> xc_psr_feat_type type, xc_psr_hw_info *hw_info);
>
> diff --git a/tools/libxc/xc_psr.c b/tools/libxc/xc_psr.c
> index 385d547..5660d51 100644
> --- a/tools/libxc/xc_psr.c
> +++ b/tools/libxc/xc_psr.c
> @@ -283,9 +283,9 @@ int xc_psr_cat_set_domain_data(xc_interface *xch, uint32_t domid,
> return do_domctl(xch, &domctl);
> }
>
> -int xc_psr_cat_get_domain_data(xc_interface *xch, uint32_t domid,
> - xc_psr_type type, uint32_t target,
> - uint64_t *data)
> +int xc_psr_get_domain_data(xc_interface *xch, uint32_t domid,
> + xc_psr_type type, uint32_t target,
> + uint64_t *data)
> {
> int rc;
> DECLARE_DOMCTL;
> @@ -305,6 +305,9 @@ int xc_psr_cat_get_domain_data(xc_interface *xch, uint32_t domid,
> case XC_PSR_CAT_L2_CBM:
> cmd = XEN_DOMCTL_PSR_GET_L2_CBM;
> break;
> + case XC_PSR_MBA_THRTL:
> + cmd = XEN_DOMCTL_PSR_GET_MBA_THRTL;
> + break;
> default:
> errno = EINVAL;
> return -1;
> diff --git a/tools/libxl/libxl_psr.c b/tools/libxl/libxl_psr.c
> index e9d36dc..5a4ba3d 100644
> --- a/tools/libxl/libxl_psr.c
> +++ b/tools/libxl/libxl_psr.c
> @@ -71,16 +71,29 @@ static void libxl__psr_cmt_log_err_msg(libxl__gc *gc, int err)
> LOGE(ERROR, "%s", msg);
> }
>
> -static void libxl__psr_cat_log_err_msg(libxl__gc *gc, int err)
> +static void libxl__psr_alloc_log_err_msg(libxl__gc *gc,
> + int err,
> + libxl_psr_type type)
> {
> + /*
> + * Index is 'libxl_psr_type' so we set two 'CDP' to correspond to
> + * DATA and CODE.
> + */
> + const char * const feat_name[] = {
> + [LIBXL_PSR_CBM_TYPE_UNKNOWN] = "UNKNOWN",
> + [LIBXL_PSR_CBM_TYPE_L3_CBM] = "L3 CAT",
> + [LIBXL_PSR_CBM_TYPE_L3_CBM_CODE...LIBXL_PSR_CBM_TYPE_L3_CBM_DATA] = "CDP",
Newline for "CDP".
> + [LIBXL_PSR_CBM_TYPE_L2_CBM] = "L2 CAT",
> + [LIBXL_PSR_CBM_TYPE_MBA_THRTL] = "MBA",
> + };
> char *msg;
>
> switch (err) {
> case ENODEV:
> - msg = "CAT is not supported in this system";
> + msg = "is not supported in this system";
> break;
> case ENOENT:
> - msg = "CAT is not enabled on the socket";
> + msg = "is not enabled on the socket";
> break;
> case EOVERFLOW:
> msg = "no free COS available";
> @@ -106,7 +119,7 @@ static void libxl__psr_cat_log_err_msg(libxl__gc *gc, int err)
> return;
> }
>
> - LOGE(ERROR, "%s", msg);
> + LOG(ERROR, "%s: %s", feat_name[type], msg);
> }
>
> static int libxl__pick_socket_cpu(libxl__gc *gc, uint32_t socketid)
> @@ -303,10 +316,10 @@ out:
> return rc;
> }
>
> -static inline xc_psr_type libxl__psr_cbm_type_to_libxc_psr_type(
> - libxl_psr_cbm_type type)
> +static inline xc_psr_type libxl__psr_type_to_libxc_psr_type(
> + libxl_psr_type type)
> {
> - BUILD_BUG_ON(sizeof(libxl_psr_cbm_type) != sizeof(xc_psr_type));
> + BUILD_BUG_ON(sizeof(libxl_psr_type) != sizeof(xc_psr_type));
> return (xc_psr_type)type;
> }
>
> @@ -325,14 +338,14 @@ int libxl_psr_cat_set_cbm(libxl_ctx *ctx, uint32_t domid,
> }
>
> libxl_for_each_set_bit(socketid, *target_map) {
> - xc_psr_type xc_type = libxl__psr_cbm_type_to_libxc_psr_type(type);;
> + xc_psr_type xc_type = libxl__psr_type_to_libxc_psr_type(type);;
Double ';' carried from previous patch.
>
> if (socketid >= nr_sockets)
> break;
>
> if (xc_psr_cat_set_domain_data(ctx->xch, domid, xc_type,
> socketid, cbm)) {
> - libxl__psr_cat_log_err_msg(gc, errno);
> + libxl__psr_alloc_log_err_msg(gc, errno, type);
> rc = ERROR_FAIL;
> }
> }
> @@ -346,18 +359,7 @@ int libxl_psr_cat_get_cbm(libxl_ctx *ctx, uint32_t domid,
> libxl_psr_cbm_type type, uint32_t target,
> uint64_t *cbm_r)
> {
> - GC_INIT(ctx);
> - int rc = 0;
> - xc_psr_type xc_type = libxl__psr_cbm_type_to_libxc_psr_type(type);
> -
> - if (xc_psr_cat_get_domain_data(ctx->xch, domid, xc_type,
> - target, cbm_r)) {
> - libxl__psr_cat_log_err_msg(gc, errno);
> - rc = ERROR_FAIL;
> - }
> -
> - GC_FREE;
> - return rc;
> + return libxl_psr_get_val(ctx, domid, type, target, cbm_r);
> }
>
> static xc_psr_feat_type libxl__feat_type_to_libxc_feat_type(
> @@ -455,7 +457,19 @@ int libxl_psr_get_val(libxl_ctx *ctx, uint32_t domid,
> libxl_psr_type type, unsigned int target,
> uint64_t *val)
> {
> - return ERROR_FAIL;
> + GC_INIT(ctx);
> + int rc = 0;
> +
Extra newline?
> + xc_psr_type xc_type = libxl__psr_type_to_libxc_psr_type(type);
> +
> + if (xc_psr_get_domain_data(ctx->xch, domid, xc_type,
> + target, val)) {
> + libxl__psr_alloc_log_err_msg(gc, errno, type);
> + rc = ERROR_FAIL;
> + }
> +
> + GC_FREE;
> + return rc;
> }
>
> static void libxl__xc_hw_info_to_libxl_hw_info(
> diff --git a/tools/xl/xl.h b/tools/xl/xl.h
> index 01c2af6..464831e 100644
> --- a/tools/xl/xl.h
> +++ b/tools/xl/xl.h
> @@ -205,6 +205,7 @@ int main_psr_cmt_detach(int argc, char **argv);
> int main_psr_cmt_show(int argc, char **argv);
> int main_psr_cat_cbm_set(int argc, char **argv);
> int main_psr_cat_show(int argc, char **argv);
> +int main_psr_mba_show(int argc, char **argv);
> #endif
> int main_qemu_monitor_command(int argc, char **argv);
>
> diff --git a/tools/xl/xl_cmdtable.c b/tools/xl/xl_cmdtable.c
> index a1679e9..3136d70 100644
> --- a/tools/xl/xl_cmdtable.c
> +++ b/tools/xl/xl_cmdtable.c
> @@ -565,6 +565,11 @@ struct cmd_spec cmd_table[] = {
> "[options] <Domain>",
> "-l <level> Specify the cache level to process, otherwise L3 cache is processed\n"
> },
> + { "psr-mba-show",
> + &main_psr_mba_show, 0, 1,
> + "Show Memory Bandwidth Allocation information",
> + "<Domain>",
> + },
> #endif
> { "usbctrl-attach",
> &main_usbctrl_attach, 0, 1,
> diff --git a/tools/xl/xl_psr.c b/tools/xl/xl_psr.c
> index 40269b4..46b7788 100644
> --- a/tools/xl/xl_psr.c
> +++ b/tools/xl/xl_psr.c
> @@ -327,19 +327,27 @@ out:
> return rc;
> }
>
> -static void psr_cat_print_one_domain_cbm_type(uint32_t domid, uint32_t socketid,
> - libxl_psr_cbm_type type)
> +static void psr_print_one_domain_val_type(uint32_t domid,
> + libxl_psr_hw_info *info,
> + libxl_psr_type type)
> {
> - uint64_t cbm;
> + uint64_t val;
>
> - if (!libxl_psr_cat_get_cbm(ctx, domid, type, socketid, &cbm))
> - printf("%#16"PRIx64, cbm);
> + if (!libxl_psr_get_val(ctx, domid, type, info->id, &val))
> + {
> + if (type == LIBXL_PSR_CBM_TYPE_MBA_THRTL && info->u.mba.linear)
> + printf("%16"PRIu64, val);
> + else
> + printf("%#16"PRIx64, val);
> + }
> else
> printf("%16s", "error");
> }
>
> -static void psr_cat_print_one_domain_cbm(uint32_t domid, uint32_t socketid,
> - bool cdp_enabled, unsigned int lvl)
> +static void psr_print_one_domain_val(uint32_t domid,
> + libxl_psr_hw_info *info,
> + libxl_psr_feat_type type,
> + unsigned int lvl)
> {
> char *domain_name;
>
> @@ -347,106 +355,154 @@ static void psr_cat_print_one_domain_cbm(uint32_t domid, uint32_t socketid,
> printf("%5d%25s", domid, domain_name);
> free(domain_name);
>
> - switch (lvl) {
> - case 3:
> - if (!cdp_enabled) {
> - psr_cat_print_one_domain_cbm_type(domid, socketid,
> + switch (type) {
> + case LIBXL_PSR_FEAT_TYPE_CAT:
> + switch (lvl) {
> + case 3:
> + if (!info->u.cat.cdp_enabled) {
> + psr_print_one_domain_val_type(domid, info,
> LIBXL_PSR_CBM_TYPE_L3_CBM);
> - } else {
> - psr_cat_print_one_domain_cbm_type(domid, socketid,
> + } else {
> + psr_print_one_domain_val_type(domid, info,
> LIBXL_PSR_CBM_TYPE_L3_CBM_CODE);
> - psr_cat_print_one_domain_cbm_type(domid, socketid,
> + psr_print_one_domain_val_type(domid, info,
> LIBXL_PSR_CBM_TYPE_L3_CBM_DATA);
> - }
> - break;
> - case 2:
> - psr_cat_print_one_domain_cbm_type(domid, socketid,
> + }
> + break;
> +
> + case 2:
> + psr_print_one_domain_val_type(domid, info,
> LIBXL_PSR_CBM_TYPE_L2_CBM);
> + break;
> +
> + default:
> + printf("Input lvl %d is wrong!", lvl);
> + }
> break;
> - default:
> - printf("Input lvl %d is wrong!", lvl);
> +
> + case LIBXL_PSR_FEAT_TYPE_MBA:
> + psr_print_one_domain_val_type(domid, info,
> + LIBXL_PSR_CBM_TYPE_MBA_THRTL);
> break;
> }
>
> printf("\n");
> }
>
> -static int psr_cat_print_domain_cbm(uint32_t domid, uint32_t socketid,
> - bool cdp_enabled, unsigned int lvl)
> +static int psr_print_domain_val(uint32_t domid,
> + libxl_psr_hw_info *info,
> + libxl_psr_feat_type type,
> + unsigned int lvl)
> {
> int i, nr_domains;
> libxl_dominfo *list;
>
> if (domid != INVALID_DOMID) {
> - psr_cat_print_one_domain_cbm(domid, socketid, cdp_enabled, lvl);
> + psr_print_one_domain_val(domid, info, type, lvl);
> return 0;
> }
>
> if (!(list = libxl_list_domain(ctx, &nr_domains))) {
> - fprintf(stderr, "Failed to get domain list for cbm display\n");
> - return -1;
> + fprintf(stderr, "Failed to get domain list for value display\n");
> + return EXIT_FAILURE;
> }
>
> for (i = 0; i < nr_domains; i++)
> - psr_cat_print_one_domain_cbm(list[i].domid, socketid, cdp_enabled, lvl);
> + psr_print_one_domain_val(list[i].domid, info, type, lvl);
> libxl_dominfo_list_free(list, nr_domains);
>
> return 0;
> }
>
> -static int psr_cat_print_socket(uint32_t domid, libxl_psr_cat_info *info,
> - unsigned int lvl)
> +static int psr_print_socket(uint32_t domid,
> + libxl_psr_hw_info *info,
> + libxl_psr_feat_type type,
> + unsigned int lvl)
> {
> - int rc;
> - uint32_t l3_cache_size;
> -
> printf("%-16s: %u\n", "Socket ID", info->id);
>
> - /* So far, CMT only supports L3 cache. */
> - if (lvl == 3) {
> - rc = libxl_psr_cmt_get_l3_cache_size(ctx, info->id, &l3_cache_size);
> - if (rc) {
> - fprintf(stderr, "Failed to get l3 cache size for socket:%d\n",
> - info->id);
> - return -1;
> + switch (type) {
> + case LIBXL_PSR_FEAT_TYPE_CAT:
> + {
> + int rc;
> + uint32_t l3_cache_size;
> +
> + /* So far, CMT only supports L3 cache. */
> + if (lvl == 3) {
> + rc = libxl_psr_cmt_get_l3_cache_size(ctx, info->id, &l3_cache_size);
> + if (rc) {
> + fprintf(stderr, "Failed to get l3 cache size for socket:%d\n",
> + info->id);
> + return -1;
> + }
> + printf("%-16s: %uKB\n", "L3 Cache", l3_cache_size);
> }
> - printf("%-16s: %uKB\n", "L3 Cache", l3_cache_size);
> +
> + printf("%-16s: %#llx\n", "Default CBM",
> + (1ull << info->u.cat.cbm_len) - 1);
> + if (info->u.cat.cdp_enabled)
> + printf("%5s%25s%16s%16s\n", "ID", "NAME", "CBM (code)", "CBM (data)");
> + else
> + printf("%5s%25s%16s\n", "ID", "NAME", "CBM");
> +
> + break;
> }
>
> - printf("%-16s: %#llx\n", "Default CBM", (1ull << info->cbm_len) - 1);
> - if (info->cdp_enabled)
> - printf("%5s%25s%16s%16s\n", "ID", "NAME", "CBM (code)", "CBM (data)");
> - else
> - printf("%5s%25s%16s\n", "ID", "NAME", "CBM");
> + case LIBXL_PSR_FEAT_TYPE_MBA:
> + printf("%-16s: %u\n", "Default THRTL", 0);
> + printf("%5s%25s%16s\n", "ID", "NAME", "THRTL");
> + break;
>
> - return psr_cat_print_domain_cbm(domid, info->id, info->cdp_enabled, lvl);
> + default:
> + fprintf(stderr, "Input feature type %d is wrong\n", type);
> + return EXIT_FAILURE;
> + }
> +
> + return psr_print_domain_val(domid, info, type, lvl);
> }
>
> -static int psr_cat_show(uint32_t domid, unsigned int lvl)
> +static int psr_val_show(uint32_t domid,
> + libxl_psr_feat_type type,
> + unsigned int lvl)
> {
> unsigned int i, nr;
> int rc;
> - libxl_psr_cat_info *info;
> + libxl_psr_hw_info *info;
>
> - if (lvl != 2 && lvl != 3) {
> - fprintf(stderr, "Input lvl %d is wrong\n", lvl);
> + switch (type) {
> + case LIBXL_PSR_FEAT_TYPE_CAT:
> + if (lvl != 2 && lvl != 3) {
> + fprintf(stderr, "Input lvl %d is wrong\n", lvl);
> + return EXIT_FAILURE;
> + }
> + break;
> +
> + case LIBXL_PSR_FEAT_TYPE_MBA:
> + if (lvl) {
> + fprintf(stderr, "Input lvl %d is wrong\n", lvl);
Maybe "Unexpected lvl parameter %d for MBA feature\n"?
There's no valid lvl value for MBA at all.
Thanks, Roger.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v4 14/15] tools: implement new generic set value interface and MBA set value command
2017-09-23 9:48 ` [PATCH v4 14/15] tools: implement new generic set value interface and MBA set " Yi Sun
@ 2017-09-26 11:39 ` Roger Pau Monné
2017-09-28 2:46 ` Yi Sun
0 siblings, 1 reply; 64+ messages in thread
From: Roger Pau Monné @ 2017-09-26 11:39 UTC (permalink / raw)
To: Yi Sun; +Cc: xen-devel, Wei Liu, Ian Jackson, Chao Peng
On Sat, Sep 23, 2017 at 09:48:23AM +0000, Yi Sun wrote:
> int libxl_psr_cat_get_cbm(libxl_ctx *ctx, uint32_t domid,
> @@ -450,7 +425,32 @@ int libxl_psr_set_val(libxl_ctx *ctx, uint32_t domid,
> libxl_psr_type type, libxl_bitmap *target_map,
> uint64_t val)
> {
> - return ERROR_FAIL;
> + GC_INIT(ctx);
> + int rc, socketid, nr_sockets;
> +
> + rc = libxl__count_physical_sockets(gc, &nr_sockets);
> + if (rc) {
> + LOG(ERROR, "failed to get system socket count");
> + goto out;
> + }
> +
> + libxl_for_each_set_bit(socketid, *target_map) {
> + xc_psr_type xc_type;
> +
> + if (socketid >= nr_sockets)
> + break;
> +
> + xc_type = libxl__psr_type_to_libxc_psr_type(type);
No need to do this for every loop, AFAICT you could define and
initialize this together with the other local variables.
> + if (xc_psr_set_domain_data(ctx->xch, domid, xc_type,
> + socketid, val)) {
> + libxl__psr_alloc_log_err_msg(gc, errno, type);
> + rc = ERROR_FAIL;
goto out; or break;
Or it is intended to not break the loop?
Thanks, Roger.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v4 15/15] docs: add MBA description in docs
2017-09-23 9:48 ` [PATCH v4 15/15] docs: add MBA description in docs Yi Sun
@ 2017-09-26 11:45 ` Roger Pau Monné
0 siblings, 0 replies; 64+ messages in thread
From: Roger Pau Monné @ 2017-09-26 11:45 UTC (permalink / raw)
To: Yi Sun; +Cc: xen-devel, Wei Liu, Ian Jackson, Chao Peng
On Sat, Sep 23, 2017 at 09:48:24AM +0000, Yi Sun wrote:
> This patch adds MBA description in related documents.
>
> Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
> Acked-by: Wei Liu <wei.liu2@citrix.com>
There's still a 'closed-loop' mention that I think I've already
pointed out in the last review, but anyway:
Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
> ---
> CC: Wei Liu <wei.liu2@citrix.com>
> CC: Ian Jackson <ian.jackson@eu.citrix.com>
> CC: Roger Pau Monné <roger.pau@citrix.com>
> CC: Chao Peng <chao.p.peng@linux.intel.com>
>
> v4:
> - modify description of MBA in 'xl.pod.1.in' to be same as feature doc.
> (suggested by Roger Pau Monné)
> - fix words issue.
> (suggested by Roger Pau Monné)
> v2:
> - state the value type shown by 'psr-mba-show'. For linear mode,
> it shows decimal value. For non-linear mode, it shows hexadecimal
> value.
> (suggested by Chao Peng)
> ---
> docs/man/xl.pod.1.in | 33 +++++++++++++++++++++++++
> docs/misc/xl-psr.markdown | 63 +++++++++++++++++++++++++++++++++++++++++++++++
> 2 files changed, 96 insertions(+)
>
> diff --git a/docs/man/xl.pod.1.in b/docs/man/xl.pod.1.in
> index 3d5f2f7..64d87ba 100644
> --- a/docs/man/xl.pod.1.in
> +++ b/docs/man/xl.pod.1.in
> @@ -1803,6 +1803,39 @@ processed.
>
> =back
>
> +=head2 Memory Bandwidth Allocation
> +
> +Intel Skylake and later server platforms offer capabilities to configure and
> +make use of the Memory Bandwidth Allocation (MBA) mechanisms, which provides
> +OS/VMMs the ability to slow misbehaving apps/VMs by using a credit-based
> +throttling mechanism. In the Xen implementation, MBA is used to control memory
> +bandwidth on VM basis. To enforce bandwidth on a specific domain, just set
> +throttling value (THRTL) for the domain.
> +
> +=over 4
> +
> +=item B<psr-mba-set> [I<OPTIONS>] I<domain-id> I<thrtl>
> +
> +Set throttling value (THRTL) for a domain. For how to specify I<thrtl>
> +please refer to L<http://xenbits.xen.org/docs/unstable/misc/xl-psr.html>.
> +
> +B<OPTIONS>
> +
> +=over 4
> +
> +=item B<-s SOCKET>, B<--socket=SOCKET>
> +
> +Specify the socket to process, otherwise all sockets are processed.
> +
> +=back
> +
> +=item B<psr-mba-show> [I<domain-id>]
> +
> +Show MBA settings for a certain domain or all domains. For linear mode, it
> +shows the decimal value. For non-linear mode, it shows hexadecimal value.
> +
> +=back
> +
> =head1 IGNORED FOR COMPATIBILITY WITH XM
>
> xl is mostly command-line compatible with the old xm utility used with
> diff --git a/docs/misc/xl-psr.markdown b/docs/misc/xl-psr.markdown
> index 04dd957..08c06dc 100644
> --- a/docs/misc/xl-psr.markdown
> +++ b/docs/misc/xl-psr.markdown
> @@ -186,6 +186,69 @@ Setting data CBM for a domain:
> Setting the same code and data CBM for a domain:
> `xl psr-cat-set <domid> <cbm>`
>
> +## Memory Bandwidth Allocation (MBA)
> +
> +Memory Bandwidth Allocation (MBA) is a new feature available on Intel
> +Skylake and later server platforms that allows an OS or Hypervisor/VMM to
> +slow misbehaving apps/VMs or create advanced closed-loop control system via
> +exposing control over a credit-based throttling mechanism. To enforce bandwidth
> +on a specific domain, just set throttling value (THRTL) into Class of Service
> +(COS). MBA provides two THRTL mode. One is linear mode and the other is
> +non-linear mode.
> +
> +In the linear mode the input precision is defined as 100-(THRTL_MAX). Values
> +not an even multiple of the precision (e.g., 12%) will be rounded down (e.g.,
> +to 10% delay by the hardware).
> +
> +If linear values are not supported then input delay values are powers-of-two
> +from zero to the THRTL_MAX value from CPUID. In this case any values not a power
> +of two will be rounded down the next nearest power of two.
> +
> +For example, assuming a system with 2 domains:
> +
> + * A THRTL of 0x0 for every domain means each domain can access the whole cache
> + without any delay. This is the default.
> +
> + * Linear mode: Giving one domain a THRTL of 0xC and the other domain's 0 means
> + that the first domain gets 10% delay to access the cache and the other one
> + without any delay.
> +
> + * Non-linear mode: Giving one domain a THRTL of 0xC and the other domain's 0
> + means that the first domain gets 8% delay to access the cache and the other
> + one without any delay.
> +
> +For more detailed information please refer to Intel SDM chapter
> +"Introduction to Memory Bandwidth Allocation".
> +
> +In Xen's implementation, THRTL can be configured with libxl/xl interfaces but
> +COS is maintained in hypervisor only. The cache partition granularity is per
> +domain, each domain has COS=0 assigned by default, the corresponding THRTL is
> +0, which means all the cache resource can be accessed without delay.
> +
> +### xl interfaces
> +
> +System MBA information such as maximum COS and maximum THRTL can be obtained by:
> +
> +`xl psr-hwinfo --mba`
> +
> +The simplest way to change a domain's THRTL from its default is running:
> +
> +`xl psr-mba-set [OPTIONS] <domid> <thrtl>`
> +
> +In a multi-socket system, the same thrtl will be set on each socket by default.
> +Per socket thrtl can be specified with the `--socket SOCKET` option.
> +
> +Setting the THRTL may not be successful if insufficient COS is available. In
^ not enough COS are available.
> +such case unused COS(es) may be freed by setting THRTL of all related domains to
> +its default value(0).
> +
> +Per domain THRTL settings can be shown by:
> +
> +`xl psr-mba-show [OPTIONS] <domid>`
> +
> +For linear mode, it shows the decimal value. For non-linear mode, it shows
> +hexadecimal value.
> +
> ## Reference
>
> [1] Intel SDM
> --
> 1.9.1
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v4 00/15] Enable Memory Bandwidth Allocation in Xen
2017-09-23 9:48 [PATCH v4 00/15] Enable Memory Bandwidth Allocation in Xen Yi Sun
` (14 preceding siblings ...)
2017-09-23 9:48 ` [PATCH v4 15/15] docs: add MBA description in docs Yi Sun
@ 2017-09-26 11:48 ` Roger Pau Monné
2017-09-28 2:20 ` Yi Sun
2017-09-28 15:57 ` Wei Liu
16 siblings, 1 reply; 64+ messages in thread
From: Roger Pau Monné @ 2017-09-26 11:48 UTC (permalink / raw)
To: Yi Sun
Cc: Wei Liu, Andrew Cooper, Ian Jackson, Julien Grall, Jan Beulich,
Chao Peng, xen-devel, Daniel De Graaf
On Sat, Sep 23, 2017 at 09:48:09AM +0000, Yi Sun wrote:
> Hi, all,
>
> We plan to bring a new PSR (Platform Shared Resource) feature called
> Intel Memory Bandwidth Allocation (MBA) to Xen.
>
> Besides the MBA enabling, we change some interfaces to make them more
> general but not only for CAT.
As a general comment, I'm quite amused by the huge amount of code
movement and name changes that's needed in order to implement this
'simple' feature. In the future it would be good if things like this
(adding another feature to an existing interface) is not so painful. I
hope that with this change all the PSR infrastructure is flexible
enough to support upcoming PSR features, in case there are more.
Thanks, Roger.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v4 05/15] x86: implement get hw info flow for MBA
2017-09-26 8:46 ` Roger Pau Monné
@ 2017-09-26 11:49 ` Jan Beulich
0 siblings, 0 replies; 64+ messages in thread
From: Jan Beulich @ 2017-09-26 11:49 UTC (permalink / raw)
To: Roger Pau Monné, Yi Sun; +Cc: AndrewCooper, Wei Liu, xen-devel, Chao Peng
>>> On 26.09.17 at 10:46, <roger.pau@citrix.com> wrote:
> On Sat, Sep 23, 2017 at 09:48:14AM +0000, Yi Sun wrote:
>> --- a/xen/arch/x86/sysctl.c
>> +++ b/xen/arch/x86/sysctl.c
>> @@ -174,7 +174,7 @@ long arch_do_sysctl(
>> case XEN_SYSCTL_psr_alloc:
>> switch ( sysctl->u.psr_alloc.cmd )
>> {
>> - uint32_t data[PSR_INFO_ARRAY_SIZE];
>> + uint32_t data[PSR_INFO_ARRAY_SIZE] = {0};
>
> I would rather do "{ 0 };" or "{ };" (note the spaces around the 0).
The latter of the two suggested variants please.
Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v4 02/15] Rename PSR sysctl/domctl interfaces and xsm policy to make them be general
2017-09-23 9:48 ` [PATCH v4 02/15] Rename PSR sysctl/domctl interfaces and xsm policy to make them be general Yi Sun
2017-09-25 16:15 ` Roger Pau Monné
@ 2017-09-26 14:19 ` Wei Liu
2017-09-28 2:12 ` Yi Sun
2017-09-28 10:21 ` Jan Beulich
2 siblings, 1 reply; 64+ messages in thread
From: Wei Liu @ 2017-09-26 14:19 UTC (permalink / raw)
To: Yi Sun
Cc: Wei Liu, Andrew Cooper, Ian Jackson, Jan Beulich, Chao Peng,
xen-devel, Daniel De Graaf, Roger Pau Monné
On Sat, Sep 23, 2017 at 05:48:11PM +0800, Yi Sun wrote:
> This patch renames PSR sysctl/domctl interfaces and related xsm policy to
> make them be general for all resource allocation features but not only
> for CAT. Then, we can resuse the interfaces for all allocation features.
>
> Basically, it changes 'psr_cat_op' to 'psr_alloc', and remove 'CAT_' from some
> macros. E.g.:
> 1. psr_cat_op -> psr_alloc
> 2. XEN_DOMCTL_psr_cat_op -> XEN_DOMCTL_psr_alloc
> 3. XEN_SYSCTL_psr_cat_op -> XEN_SYSCTL_psr_alloc
> 4. XEN_DOMCTL_PSR_CAT_SET_L3_CBM -> XEN_DOMCTL_PSR_SET_L3_CBM
> 5. XEN_SYSCTL_PSR_CAT_get_l3_info -> XEN_SYSCTL_PSR_get_l3_info
>
> The sysctl/domctl version numbers are bumped.
>
> Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
> Reviewed-by: Wei Liu <wei.liu2@citrix.com>
> Acked-by: Daniel De Graaf <dgdegra@tycho.nsa.gov>
Normally if there have been substantive changes you should drop the
reviews. It seems to be the case. I will need to review this patch
again.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v4 02/15] Rename PSR sysctl/domctl interfaces and xsm policy to make them be general
2017-09-26 14:19 ` Wei Liu
@ 2017-09-28 2:12 ` Yi Sun
2017-09-28 8:34 ` Wei Liu
0 siblings, 1 reply; 64+ messages in thread
From: Yi Sun @ 2017-09-28 2:12 UTC (permalink / raw)
To: Wei Liu
Cc: Andrew Cooper, Ian Jackson, Jan Beulich, Chao Peng, xen-devel,
Daniel De Graaf, Roger Pau Monn�
On 17-09-26 15:19:19, Wei Liu wrote:
> On Sat, Sep 23, 2017 at 05:48:11PM +0800, Yi Sun wrote:
> > This patch renames PSR sysctl/domctl interfaces and related xsm policy to
> > make them be general for all resource allocation features but not only
> > for CAT. Then, we can resuse the interfaces for all allocation features.
> >
> > Basically, it changes 'psr_cat_op' to 'psr_alloc', and remove 'CAT_' from some
> > macros. E.g.:
> > 1. psr_cat_op -> psr_alloc
> > 2. XEN_DOMCTL_psr_cat_op -> XEN_DOMCTL_psr_alloc
> > 3. XEN_SYSCTL_psr_cat_op -> XEN_SYSCTL_psr_alloc
> > 4. XEN_DOMCTL_PSR_CAT_SET_L3_CBM -> XEN_DOMCTL_PSR_SET_L3_CBM
> > 5. XEN_SYSCTL_PSR_CAT_get_l3_info -> XEN_SYSCTL_PSR_get_l3_info
> >
> > The sysctl/domctl version numbers are bumped.
> >
> > Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
> > Reviewed-by: Wei Liu <wei.liu2@citrix.com>
> > Acked-by: Daniel De Graaf <dgdegra@tycho.nsa.gov>
>
> Normally if there have been substantive changes you should drop the
> reviews. It seems to be the case. I will need to review this patch
> again.
Sorry for that.
I thought they were just name changes. So I did not drop reviews. But
this patch's main purpose is 'rename'. I should drop reviews. Will
notice this.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v4 00/15] Enable Memory Bandwidth Allocation in Xen
2017-09-26 11:48 ` [PATCH v4 00/15] Enable Memory Bandwidth Allocation in Xen Roger Pau Monné
@ 2017-09-28 2:20 ` Yi Sun
0 siblings, 0 replies; 64+ messages in thread
From: Yi Sun @ 2017-09-28 2:20 UTC (permalink / raw)
To: Roger Pau Monn�
Cc: Wei Liu, Andrew Cooper, Ian Jackson, Julien Grall, Jan Beulich,
Chao Peng, xen-devel, Daniel De Graaf
On 17-09-26 12:48:29, Roger Pau Monn� wrote:
> On Sat, Sep 23, 2017 at 09:48:09AM +0000, Yi Sun wrote:
> > Hi, all,
> >
> > We plan to bring a new PSR (Platform Shared Resource) feature called
> > Intel Memory Bandwidth Allocation (MBA) to Xen.
> >
> > Besides the MBA enabling, we change some interfaces to make them more
> > general but not only for CAT.
>
> As a general comment, I'm quite amused by the huge amount of code
> movement and name changes that's needed in order to implement this
> 'simple' feature. In the future it would be good if things like this
> (adding another feature to an existing interface) is not so painful. I
> hope that with this change all the PSR infrastructure is flexible
> enough to support upcoming PSR features, in case there are more.
>
I understand your concerns. The original codes are specificlly designed for
only L3 CAT. So, it is not easy to add new feature. That is why I refactored
PSR codes to provide a extendible framework in hypervisor through L2 CAT patch
set. But L2 CAT is also a CAT feature. When the first versions of L2 CAT patch
set were submitted, the MBA was not published yet. To avoid confusion to the
codes and extra argument, I kept many old names and interfaces in tools.
Now, MBA has been published. So, I made such 'huge amount of code movement and
name changes' to try to cover all PSR features. The purpose is to provide an
extendible framework and generic interfaces.
> Thanks, Roger.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v4 07/15] x86: implement set value flow for MBA
2017-09-26 9:39 ` Roger Pau Monné
@ 2017-09-28 2:39 ` Yi Sun
2017-09-28 11:15 ` Jan Beulich
0 siblings, 1 reply; 64+ messages in thread
From: Yi Sun @ 2017-09-28 2:39 UTC (permalink / raw)
To: Roger Pau Monn�
Cc: xen-devel, Wei Liu, Chao Peng, Jan Beulich, Andrew Cooper
On 17-09-26 10:39:31, Roger Pau Monn� wrote:
> On Sat, Sep 23, 2017 at 09:48:16AM +0000, Yi Sun wrote:
> > @@ -274,29 +277,6 @@ static enum psr_feat_type psr_type_to_feat_type(enum psr_type type)
> > return feat_type;
> > }
> >
> > -static bool psr_check_cbm(unsigned int cbm_len, unsigned long cbm)
> > -{
> > - unsigned int first_bit, zero_bit;
> > -
> > - /* Set bits should only in the range of [0, cbm_len]. */
> > - if ( cbm & (~0ul << cbm_len) )
> > - return false;
> > -
> > - /* At least one bit need to be set. */
> > - if ( cbm == 0 )
> > - return false;
> > -
> > - first_bit = find_first_bit(&cbm, cbm_len);
> > - zero_bit = find_next_zero_bit(&cbm, cbm_len, first_bit);
> > -
> > - /* Set bits should be contiguous. */
> > - if ( zero_bit < cbm_len &&
> > - find_next_bit(&cbm, cbm_len, zero_bit) < cbm_len )
> > - return false;
> > -
> > - return true;
> > -}
> > -
> > /* Implementation of allocation features' functions. */
> > static bool cat_init_feature(const struct cpuid_leaf *regs,
> > struct feat_node *feat,
> > @@ -426,11 +406,36 @@ static bool cat_get_feat_info(const struct feat_node *feat,
> > return true;
> > }
> >
> > +static bool cat_check_cbm(const struct feat_node *feat, unsigned long cbm)
> > +{
> > + unsigned int first_bit, zero_bit;
> > + unsigned int cbm_len = feat->cat.cbm_len;
> > +
> > + /*
> > + * Set bits should only in the range of [0, cbm_len].
> > + * And, at least one bit need to be set.
> > + */
> > + if ( cbm & (~0ul << cbm_len) || cbm == 0 )
> > + return false;
> > +
> > + first_bit = find_first_bit(&cbm, cbm_len);
> > + zero_bit = find_next_zero_bit(&cbm, cbm_len, first_bit);
> > +
> > + /* Set bits should be contiguous. */
> > + if ( zero_bit < cbm_len &&
> > + find_next_bit(&cbm, cbm_len, zero_bit) < cbm_len )
> > + return false;
> > +
> > + return true;
> > +}
>
> Why do you need to move the code apart from renaming it?
>
Because it is CAT specific function now. I moved it into CAT section.
'/* Implementation of allocation features' functions. */'
> > @@ -1210,25 +1237,39 @@ static unsigned int get_socket_cpu(unsigned int socket)
> > struct cos_write_info
> > {
> > unsigned int cos;
> > - struct feat_node *feature;
> > + struct feat_node **features;
> > const uint32_t *val;
> > - const struct feat_props *props;
> > + unsigned int array_len;
> > + const struct feat_props **props;
>
> Why do you need props here, from the usage below it's just pointing
> to feat_props, which is already available in this context.
>
I may drop it.
> > };
> >
> > static void do_write_psr_msrs(void *data)
> > {
> > const struct cos_write_info *info = data;
> > - struct feat_node *feat = info->feature;
> > - const struct feat_props *props = info->props;
> > - unsigned int i, cos = info->cos, cos_num = props->cos_num;
> > + unsigned int i, index = 0, cos = info->cos;
> > + const uint32_t *val_array = info->val;
> >
> > - for ( i = 0; i < cos_num; i++ )
> > + for ( i = 0; i < ARRAY_SIZE(feat_props); i++ )
> > {
> > - if ( feat->cos_reg_val[cos * cos_num + i] != info->val[i] )
> > + struct feat_node *feat = info->features[i];
> > + const struct feat_props *props = info->props[i];
>
> If you use ARRAY_SIZE(feat_props), the above should be feat_props[i].
Ok.
> Also I'm worried about the size of the props array, isn't there a
> possibility that the props array is smaller than the feature array?
>
No, every member is inserted into props array if the feature is initialized
successfully and inserted into feature array. So, they are 1:1.
>
> Thanks, Roger.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v4 14/15] tools: implement new generic set value interface and MBA set value command
2017-09-26 11:39 ` Roger Pau Monné
@ 2017-09-28 2:46 ` Yi Sun
0 siblings, 0 replies; 64+ messages in thread
From: Yi Sun @ 2017-09-28 2:46 UTC (permalink / raw)
To: Roger Pau Monn�; +Cc: xen-devel, Wei Liu, Ian Jackson, Chao Peng
On 17-09-26 12:39:21, Roger Pau Monn� wrote:
> On Sat, Sep 23, 2017 at 09:48:23AM +0000, Yi Sun wrote:
> > int libxl_psr_cat_get_cbm(libxl_ctx *ctx, uint32_t domid,
> > @@ -450,7 +425,32 @@ int libxl_psr_set_val(libxl_ctx *ctx, uint32_t domid,
> > libxl_psr_type type, libxl_bitmap *target_map,
> > uint64_t val)
> > {
> > - return ERROR_FAIL;
> > + GC_INIT(ctx);
> > + int rc, socketid, nr_sockets;
> > +
> > + rc = libxl__count_physical_sockets(gc, &nr_sockets);
> > + if (rc) {
> > + LOG(ERROR, "failed to get system socket count");
> > + goto out;
> > + }
> > +
> > + libxl_for_each_set_bit(socketid, *target_map) {
> > + xc_psr_type xc_type;
> > +
> > + if (socketid >= nr_sockets)
> > + break;
> > +
> > + xc_type = libxl__psr_type_to_libxc_psr_type(type);
>
> No need to do this for every loop, AFAICT you could define and
> initialize this together with the other local variables.
>
Thanks!
> > + if (xc_psr_set_domain_data(ctx->xch, domid, xc_type,
> > + socketid, val)) {
> > + libxl__psr_alloc_log_err_msg(gc, errno, type);
> > + rc = ERROR_FAIL;
> goto out; or break;
>
> Or it is intended to not break the loop?
>
Yes, I intended to try to set value on next socket.
> Thanks, Roger.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v4 02/15] Rename PSR sysctl/domctl interfaces and xsm policy to make them be general
2017-09-28 2:12 ` Yi Sun
@ 2017-09-28 8:34 ` Wei Liu
0 siblings, 0 replies; 64+ messages in thread
From: Wei Liu @ 2017-09-28 8:34 UTC (permalink / raw)
To: Yi Sun
Cc: Wei Liu, Andrew Cooper, Ian Jackson, Jan Beulich, Chao Peng,
xen-devel, Daniel De Graaf, Roger Pau Monn�
On Thu, Sep 28, 2017 at 10:12:08AM +0800, Yi Sun wrote:
> On 17-09-26 15:19:19, Wei Liu wrote:
> > On Sat, Sep 23, 2017 at 05:48:11PM +0800, Yi Sun wrote:
> > > This patch renames PSR sysctl/domctl interfaces and related xsm policy to
> > > make them be general for all resource allocation features but not only
> > > for CAT. Then, we can resuse the interfaces for all allocation features.
> > >
> > > Basically, it changes 'psr_cat_op' to 'psr_alloc', and remove 'CAT_' from some
> > > macros. E.g.:
> > > 1. psr_cat_op -> psr_alloc
> > > 2. XEN_DOMCTL_psr_cat_op -> XEN_DOMCTL_psr_alloc
> > > 3. XEN_SYSCTL_psr_cat_op -> XEN_SYSCTL_psr_alloc
> > > 4. XEN_DOMCTL_PSR_CAT_SET_L3_CBM -> XEN_DOMCTL_PSR_SET_L3_CBM
> > > 5. XEN_SYSCTL_PSR_CAT_get_l3_info -> XEN_SYSCTL_PSR_get_l3_info
> > >
> > > The sysctl/domctl version numbers are bumped.
> > >
> > > Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
> > > Reviewed-by: Wei Liu <wei.liu2@citrix.com>
> > > Acked-by: Daniel De Graaf <dgdegra@tycho.nsa.gov>
> >
> > Normally if there have been substantive changes you should drop the
> > reviews. It seems to be the case. I will need to review this patch
> > again.
>
> Sorry for that.
>
> I thought they were just name changes. So I did not drop reviews. But
> this patch's main purpose is 'rename'. I should drop reviews. Will
> notice this.
No need to be sorry, really. This isn't something that can be documented
clearly.
If unsure, always drop the tags in the future.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v4 02/15] Rename PSR sysctl/domctl interfaces and xsm policy to make them be general
2017-09-23 9:48 ` [PATCH v4 02/15] Rename PSR sysctl/domctl interfaces and xsm policy to make them be general Yi Sun
2017-09-25 16:15 ` Roger Pau Monné
2017-09-26 14:19 ` Wei Liu
@ 2017-09-28 10:21 ` Jan Beulich
2017-09-29 1:34 ` Yi Sun
2 siblings, 1 reply; 64+ messages in thread
From: Jan Beulich @ 2017-09-28 10:21 UTC (permalink / raw)
To: Yi Sun
Cc: Wei Liu, Andrew Cooper, Ian Jackson, xen-devel, Chao Peng,
Daniel De Graaf, Roger Pau Monné
>>> On 23.09.17 at 11:48, <yi.y.sun@linux.intel.com> wrote:
> --- a/xen/include/public/domctl.h
> +++ b/xen/include/public/domctl.h
> @@ -37,7 +37,7 @@
> #include "hvm/save.h"
> #include "memory.h"
>
> -#define XEN_DOMCTL_INTERFACE_VERSION 0x0000000e
> +#define XEN_DOMCTL_INTERFACE_VERSION 0x0000000f
This is needed only if the series doesn't make 4.10; 4.9 had it
ad 0x0000000d.
> @@ -743,22 +743,22 @@ struct xen_sysctl_pcitopoinfo {
> typedef struct xen_sysctl_pcitopoinfo xen_sysctl_pcitopoinfo_t;
> DEFINE_XEN_GUEST_HANDLE(xen_sysctl_pcitopoinfo_t);
>
> -#define XEN_SYSCTL_PSR_CAT_get_l3_info 0
> -#define XEN_SYSCTL_PSR_CAT_get_l2_info 1
> -struct xen_sysctl_psr_cat_op {
> - uint32_t cmd; /* IN: XEN_SYSCTL_PSR_CAT_* */
> +#define XEN_SYSCTL_PSR_get_l3_info 0
> +#define XEN_SYSCTL_PSR_get_l2_info 1
> +struct xen_sysctl_psr_alloc {
> + uint32_t cmd; /* IN: XEN_SYSCTL_PSR_* */
> uint32_t target; /* IN */
> union {
> struct {
> uint32_t cbm_len; /* OUT: CBM length */
> uint32_t cos_max; /* OUT: Maximum COS */
> -#define XEN_SYSCTL_PSR_CAT_L3_CDP (1u << 0)
> +#define XEN_SYSCTL_PSR_L3_CDP (1u << 0)
Is MBA have something like CDP as well, which can go through the
same subop? Otherwise I think CAT would better remain present
in this constant's name.
Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v4 03/15] x86: rename 'cbm_type' to 'psr_type' to make it general
2017-09-23 9:48 ` [PATCH v4 03/15] x86: rename 'cbm_type' to 'psr_type' to make it general Yi Sun
@ 2017-09-28 10:25 ` Jan Beulich
2017-09-29 1:35 ` Yi Sun
0 siblings, 1 reply; 64+ messages in thread
From: Jan Beulich @ 2017-09-28 10:25 UTC (permalink / raw)
To: Yi Sun; +Cc: Andrew Cooper, xen-devel, Wei Liu, Chao Peng, Roger Pau Monné
>>> On 23.09.17 at 11:48, <yi.y.sun@linux.intel.com> wrote:
> This patch renames 'cbm_type' to 'psr_type' to generalize it.
> Then, we can reuse this for all psr allocation features.
>
> Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
> Reviewed-by: Wei Liu <wei.liu2@citrix.com>
> Reviewed-by: Roger Pau Monn? <roger.pau@citrix.com>
Please could you try to get the name right here (so the committer
won't have to fiddle with it); clearly your mailer has no problem
with this, as can be seen ...
> ---
> CC: Jan Beulich <jbeulich@suse.com>
> CC: Andrew Cooper <andrew.cooper3@citrix.com>
> CC: Wei Liu <wei.liu2@citrix.com>
> CC: Roger Pau Monné <roger.pau@citrix.com>
... here.
Acked-by: Jan Beulich <jbeulich@suse.com>
Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v4 04/15] x86: implement data structure and CPU init flow for MBA
2017-09-26 8:38 ` Roger Pau Monné
@ 2017-09-28 10:27 ` Jan Beulich
0 siblings, 0 replies; 64+ messages in thread
From: Jan Beulich @ 2017-09-28 10:27 UTC (permalink / raw)
To: Roger Pau Monné, Yi Sun; +Cc: AndrewCooper, Wei Liu, xen-devel, Chao Peng
>>> On 26.09.17 at 10:38, <roger.pau@citrix.com> wrote:
> On Sat, Sep 23, 2017 at 09:48:13AM +0000, Yi Sun wrote:
>> @@ -332,20 +355,58 @@ static int cat_init_feature(const struct cpuid_leaf *regs,
>> }
>>
>> default:
>> - return -ENOENT;
>> + return false;
>> }
>>
>> /* Add this feature into array. */
>> info->features[type] = feat;
>>
>> if ( !opt_cpu_info )
>> - return 0;
>> + return true;
>>
>> printk(XENLOG_INFO "%s: enabled on socket %u, cos_max:%u, cbm_len:%u\n",
>> cat_feat_name[type], cpu_to_socket(smp_processor_id()),
>> - feat->cos_max, feat->cbm_len);
>> + feat->cos_max, feat->cat.cbm_len);
>
> I would rather do:
>
> if ( opt_cpu_info )
> printk(...);
>
> return true;
>
> So that the function has a single return path for the success case.
But not in this patch, which only partly changes what's already
there. Yet fundamentally I agree.
>> - return 0;
>> + return true;
>> +}
>> +
>> +static bool mba_init_feature(const struct cpuid_leaf *regs,
>> + struct feat_node *feat,
>> + struct psr_socket_info *info,
>> + enum psr_feat_type type)
>> +{
>> + /* No valid value so do not enable feature. */
>> + if ( !regs->a || !regs->d || type != FEAT_TYPE_MBA )
>> + return false;
>> +
>> + feat->cos_max = min(opt_cos_max, regs->d & CAT_COS_MAX_MASK);
>> + if ( feat->cos_max < 1 )
>> + return false;
>> +
>> + feat->mba.thrtl_max = (regs->a & MBA_THRTL_MAX_MASK) + 1;
>> +
>> + if ( regs->c & MBA_LINEAR_MASK )
>> + {
>> + feat->mba.linear = true;
>> +
>> + if ( feat->mba.thrtl_max >= 100 )
>> + return false;
>> + }
>> +
>> + wrmsrl(MSR_IA32_PSR_MBA_MASK(0), 0);
>> +
>> + /* Add this feature into array. */
>> + info->features[type] = feat;
>> +
>> + if ( !opt_cpu_info )
>> + return true;
>> +
>> + printk(XENLOG_INFO "MBA: enabled on socket %u, cos_max:%u, thrtl_max:%u, linear:%u.\n",
> ^ newline.
And no full stop please.
Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v4 04/15] x86: implement data structure and CPU init flow for MBA
2017-09-23 9:48 ` [PATCH v4 04/15] x86: implement data structure and CPU init flow for MBA Yi Sun
2017-09-26 8:38 ` Roger Pau Monné
@ 2017-09-28 11:00 ` Jan Beulich
2017-09-29 1:53 ` Yi Sun
1 sibling, 1 reply; 64+ messages in thread
From: Jan Beulich @ 2017-09-28 11:00 UTC (permalink / raw)
To: Yi Sun; +Cc: Andrew Cooper, xen-devel, Wei Liu, Chao Peng, Roger Pau Monné
>>> On 23.09.17 at 11:48, <yi.y.sun@linux.intel.com> wrote:
> This patch implements main data structures of MBA.
>
> Like CAT features, MBA HW info has cos_max which means the max thrtl
> register number, and thrtl_max which means the max throttle value
> (delay value). It also has a flag to represent if the throttle
> value is linear or not.
Could you replace "or not" with what the alternative actually is,
as "non-linear" can mean all sorts of things?
> One thrtl register of MBA stores a throttle value for one or more
> domains. The throttle value means the delay between L2 cache and next
> cache level.
What is a delay between two cache levels?
> @@ -272,8 +293,8 @@ static bool psr_check_cbm(unsigned int cbm_len, unsigned long cbm)
> return true;
> }
>
> -/* CAT common functions implementation. */
> -static int cat_init_feature(const struct cpuid_leaf *regs,
> +/* Implementation of allocation features' functions. */
> +static bool cat_init_feature(const struct cpuid_leaf *regs,
Such a type change should happen in a separate patch, as this
isn't specific to MBA. That way you can also make clear why you
want this to change - the current description doesn't mention
this at all.
> +static bool mba_init_feature(const struct cpuid_leaf *regs,
> + struct feat_node *feat,
> + struct psr_socket_info *info,
> + enum psr_feat_type type)
> +{
> + /* No valid value so do not enable feature. */
> + if ( !regs->a || !regs->d || type != FEAT_TYPE_MBA )
> + return false;
> +
> + feat->cos_max = min(opt_cos_max, regs->d & CAT_COS_MAX_MASK);
> + if ( feat->cos_max < 1 )
> + return false;
> +
> + feat->mba.thrtl_max = (regs->a & MBA_THRTL_MAX_MASK) + 1;
> +
> + if ( regs->c & MBA_LINEAR_MASK )
> + {
> + feat->mba.linear = true;
> +
> + if ( feat->mba.thrtl_max >= 100 )
> + return false;
> + }
> +
> + wrmsrl(MSR_IA32_PSR_MBA_MASK(0), 0);
> +
> + /* Add this feature into array. */
> + info->features[type] = feat;
> +
> + if ( !opt_cpu_info )
> + return true;
> +
> + printk(XENLOG_INFO "MBA: enabled on socket %u, cos_max:%u, thrtl_max:%u, linear:%u.\n",
The last one wants to be %d.
> @@ -1410,6 +1496,7 @@ static void psr_cpu_init(void)
> unsigned int socket, cpu = smp_processor_id();
> struct feat_node *feat;
> struct cpuid_leaf regs;
> + uint32_t ebx;
Is this local variable really a big help? To me it looks like it only
makes the patch larger without actually improving anything,
and without being related to the subject of the patch.
Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v4 06/15] x86: implement get value interface for MBA
2017-09-23 9:48 ` [PATCH v4 06/15] x86: implement get value interface " Yi Sun
@ 2017-09-28 11:12 ` Jan Beulich
0 siblings, 0 replies; 64+ messages in thread
From: Jan Beulich @ 2017-09-28 11:12 UTC (permalink / raw)
To: Yi Sun; +Cc: Andrew Cooper, xen-devel, Wei Liu, Chao Peng, Roger Pau Monné
>>> On 23.09.17 at 11:48, <yi.y.sun@linux.intel.com> wrote:
> This patch implements get value domctl interface for MBA.
>
> Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
> Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v4 07/15] x86: implement set value flow for MBA
2017-09-28 2:39 ` Yi Sun
@ 2017-09-28 11:15 ` Jan Beulich
0 siblings, 0 replies; 64+ messages in thread
From: Jan Beulich @ 2017-09-28 11:15 UTC (permalink / raw)
To: Roger Pau Monn�, Yi Sun
Cc: Andrew Cooper, Wei Liu, xen-devel, Chao Peng
>>> On 28.09.17 at 04:39, <yi.y.sun@linux.intel.com> wrote:
> On 17-09-26 10:39:31, Roger Pau Monn wrote:
>> On Sat, Sep 23, 2017 at 09:48:16AM +0000, Yi Sun wrote:
>> > @@ -274,29 +277,6 @@ static enum psr_feat_type psr_type_to_feat_type(enum psr_type type)
>> > return feat_type;
>> > }
>> >
>> > -static bool psr_check_cbm(unsigned int cbm_len, unsigned long cbm)
>> > -{
>> > - unsigned int first_bit, zero_bit;
>> > -
>> > - /* Set bits should only in the range of [0, cbm_len]. */
>> > - if ( cbm & (~0ul << cbm_len) )
>> > - return false;
>> > -
>> > - /* At least one bit need to be set. */
>> > - if ( cbm == 0 )
>> > - return false;
>> > -
>> > - first_bit = find_first_bit(&cbm, cbm_len);
>> > - zero_bit = find_next_zero_bit(&cbm, cbm_len, first_bit);
>> > -
>> > - /* Set bits should be contiguous. */
>> > - if ( zero_bit < cbm_len &&
>> > - find_next_bit(&cbm, cbm_len, zero_bit) < cbm_len )
>> > - return false;
>> > -
>> > - return true;
>> > -}
>> > -
>> > /* Implementation of allocation features' functions. */
>> > static bool cat_init_feature(const struct cpuid_leaf *regs,
>> > struct feat_node *feat,
>> > @@ -426,11 +406,36 @@ static bool cat_get_feat_info(const struct feat_node *feat,
>> > return true;
>> > }
>> >
>> > +static bool cat_check_cbm(const struct feat_node *feat, unsigned long cbm)
>> > +{
>> > + unsigned int first_bit, zero_bit;
>> > + unsigned int cbm_len = feat->cat.cbm_len;
>> > +
>> > + /*
>> > + * Set bits should only in the range of [0, cbm_len].
>> > + * And, at least one bit need to be set.
>> > + */
>> > + if ( cbm & (~0ul << cbm_len) || cbm == 0 )
>> > + return false;
>> > +
>> > + first_bit = find_first_bit(&cbm, cbm_len);
>> > + zero_bit = find_next_zero_bit(&cbm, cbm_len, first_bit);
>> > +
>> > + /* Set bits should be contiguous. */
>> > + if ( zero_bit < cbm_len &&
>> > + find_next_bit(&cbm, cbm_len, zero_bit) < cbm_len )
>> > + return false;
>> > +
>> > + return true;
>> > +}
>>
>> Why do you need to move the code apart from renaming it?
>>
> Because it is CAT specific function now. I moved it into CAT section.
> '/* Implementation of allocation features' functions. */'
The easier thing then would have been to move the comment
line up. That way actual changes to the function also become
obvious. Besides that - isn't the A in MBA also "Allocation"?
Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v4 07/15] x86: implement set value flow for MBA
2017-09-23 9:48 ` [PATCH v4 07/15] x86: implement set value flow " Yi Sun
2017-09-26 9:39 ` Roger Pau Monné
@ 2017-09-28 11:36 ` Jan Beulich
2017-09-29 2:56 ` Yi Sun
1 sibling, 1 reply; 64+ messages in thread
From: Jan Beulich @ 2017-09-28 11:36 UTC (permalink / raw)
To: Yi Sun; +Cc: Andrew Cooper, xen-devel, Wei Liu, Chao Peng, Roger Pau Monné
>>> On 23.09.17 at 11:48, <yi.y.sun@linux.intel.com> wrote:
> This patch implements set value flow for MBA including its callback
> function and domctl interface.
>
> It also changes the memebers in 'cos_write_info' to transfer the
> feature array, feature properties array and value array. Then, we
> can write all features values on the cos id into MSRs.
>
> Because multiple features may co-exist, we need handle all features to write
> values of them into a COS register with new COS ID. E.g:
> 1. L3 CAT and MBA co-exist.
> 2. Dom1 and Dom2 share a same COS ID (2). The L3 CAT CBM of Dom1 is 0x1ff,
> the MBA Thrtle of Dom1 is 0xa.
> 3. User wants to change MBA Thrtl of Dom1 to be 0x14. Because COS ID 2 is
> used by Dom2 too, we have to pick a new COS ID 3. The values of Dom1 on
> COS ID 3 are all default values as below:
> ---------
> | COS 3 |
> ---------
> L3 CAT | 0x7ff |
> ---------
> MBA | 0x0 |
> ---------
> 4. After setting, the L3 CAT CBM value of Dom1 should be kept and the new MBA
> Thrtl is set. So, the values on COS ID 3 should be below.
> ---------
> | COS 3 |
> ---------
> L3 CAT | 0x1ff |
> ---------
> MBA | 0x14 |
> ---------
>
> So, we should write all features values into their MSRs. That requires the
> feature array, feature properties array and value array are input.
How is this last aspect (and the respective changes) related to MBA?
I.e. why isn't this needed with the (also independent but possibly
co-existing) L2/L3 CAT features?
> --- a/xen/arch/x86/psr.c
> +++ b/xen/arch/x86/psr.c
> @@ -137,7 +137,10 @@ static const struct feat_props {
> uint32_t data[], unsigned int array_len);
>
> /* write_msr is used to write out feature MSR register. */
> - void (*write_msr)(unsigned int cos, uint32_t val, enum psr_type type);
> + uint32_t (*write_msr)(unsigned int cos, uint32_t val, enum psr_type type);
Again the type change would better be a prereq patch, to allow the
focus here to be on MBA.
> @@ -502,9 +514,23 @@ static bool mba_get_feat_info(const struct feat_node *feat,
> return true;
> }
>
> -static void mba_write_msr(unsigned int cos, uint32_t val,
> - enum psr_type type)
> +static uint32_t mba_write_msr(unsigned int cos, uint32_t val,
> + enum psr_type type)
> +{
> + wrmsrl(MSR_IA32_PSR_MBA_MASK(cos), val);
> +
> + /* Read actual value set by hardware. */
> + rdmsrl(MSR_IA32_PSR_MBA_MASK(cos), val);
> +
> + return val;
> +}
> +
> +static bool mba_check_thrtl(const struct feat_node *feat, unsigned long thrtl)
> {
> + if ( thrtl > feat->mba.thrtl_max )
> + return false;
> +
> + return true;
This can be had with a single return statement.
> static void do_write_psr_msrs(void *data)
> {
> const struct cos_write_info *info = data;
> - struct feat_node *feat = info->feature;
> - const struct feat_props *props = info->props;
> - unsigned int i, cos = info->cos, cos_num = props->cos_num;
> + unsigned int i, index = 0, cos = info->cos;
> + const uint32_t *val_array = info->val;
>
> - for ( i = 0; i < cos_num; i++ )
> + for ( i = 0; i < ARRAY_SIZE(feat_props); i++ )
> {
> - if ( feat->cos_reg_val[cos * cos_num + i] != info->val[i] )
> + struct feat_node *feat = info->features[i];
> + const struct feat_props *props = info->props[i];
> + unsigned int cos_num, j;
> +
> + if ( !feat || !props )
> + continue;
> +
> + cos_num = props->cos_num;
> + if ( info->array_len < index + cos_num )
> + return;
> +
> + for ( j = 0; j < cos_num; j++ )
> {
> - feat->cos_reg_val[cos * cos_num + i] = info->val[i];
> - props->write_msr(cos, info->val[i], props->type[i]);
> + if ( feat->cos_reg_val[cos * cos_num + j] != val_array[index + j] )
> + feat->cos_reg_val[cos * cos_num + j] =
> + props->write_msr(cos, val_array[index + j], props->type[j]);
This renders partly useless the check: If hardware can alter the
value, repeatedly requesting the same value to be written will
no longer guarantee the MSR write to be skipped. If hardware
behavior can't be predicted you may want to consider recording
both the value in found by reading back the register written and
the value that was written - a match with either would eliminate
the need to do the write.
Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v4 00/15] Enable Memory Bandwidth Allocation in Xen
2017-09-23 9:48 [PATCH v4 00/15] Enable Memory Bandwidth Allocation in Xen Yi Sun
` (15 preceding siblings ...)
2017-09-26 11:48 ` [PATCH v4 00/15] Enable Memory Bandwidth Allocation in Xen Roger Pau Monné
@ 2017-09-28 15:57 ` Wei Liu
2017-09-29 2:58 ` Yi Sun
16 siblings, 1 reply; 64+ messages in thread
From: Wei Liu @ 2017-09-28 15:57 UTC (permalink / raw)
To: Yi Sun
Cc: Wei Liu, Andrew Cooper, Ian Jackson, Julien Grall, Jan Beulich,
Chao Peng, xen-devel, Daniel De Graaf, Roger Pau Monné
On Sat, Sep 23, 2017 at 05:48:09PM +0800, Yi Sun wrote:
> Hi, all,
>
> We plan to bring a new PSR (Platform Shared Resource) feature called
> Intel Memory Bandwidth Allocation (MBA) to Xen.
>
> Besides the MBA enabling, we change some interfaces to make them more
> general but not only for CAT.
I will get to this series next week. You already collected a bunch of
comments, if you have addressed all of them, feel free to resend.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v4 02/15] Rename PSR sysctl/domctl interfaces and xsm policy to make them be general
2017-09-28 10:21 ` Jan Beulich
@ 2017-09-29 1:34 ` Yi Sun
2017-10-04 5:47 ` Jan Beulich
0 siblings, 1 reply; 64+ messages in thread
From: Yi Sun @ 2017-09-29 1:34 UTC (permalink / raw)
To: Jan Beulich
Cc: Wei Liu, Andrew Cooper, Ian Jackson, xen-devel, Chao Peng,
Daniel De Graaf, Roger Pau Monné
On 17-09-28 04:21:10, Jan Beulich wrote:
> >>> On 23.09.17 at 11:48, <yi.y.sun@linux.intel.com> wrote:
> > --- a/xen/include/public/domctl.h
> > +++ b/xen/include/public/domctl.h
> > @@ -37,7 +37,7 @@
> > #include "hvm/save.h"
> > #include "memory.h"
> >
> > -#define XEN_DOMCTL_INTERFACE_VERSION 0x0000000e
> > +#define XEN_DOMCTL_INTERFACE_VERSION 0x0000000f
>
> This is needed only if the series doesn't make 4.10; 4.9 had it
> ad 0x0000000d.
>
Thanks! So, this version number is upgraded only for one release. I thought
every interface change should bump it.
> > @@ -743,22 +743,22 @@ struct xen_sysctl_pcitopoinfo {
> > typedef struct xen_sysctl_pcitopoinfo xen_sysctl_pcitopoinfo_t;
> > DEFINE_XEN_GUEST_HANDLE(xen_sysctl_pcitopoinfo_t);
> >
> > -#define XEN_SYSCTL_PSR_CAT_get_l3_info 0
> > -#define XEN_SYSCTL_PSR_CAT_get_l2_info 1
> > -struct xen_sysctl_psr_cat_op {
> > - uint32_t cmd; /* IN: XEN_SYSCTL_PSR_CAT_* */
> > +#define XEN_SYSCTL_PSR_get_l3_info 0
> > +#define XEN_SYSCTL_PSR_get_l2_info 1
> > +struct xen_sysctl_psr_alloc {
> > + uint32_t cmd; /* IN: XEN_SYSCTL_PSR_* */
> > uint32_t target; /* IN */
> > union {
> > struct {
> > uint32_t cbm_len; /* OUT: CBM length */
> > uint32_t cos_max; /* OUT: Maximum COS */
> > -#define XEN_SYSCTL_PSR_CAT_L3_CDP (1u << 0)
> > +#define XEN_SYSCTL_PSR_L3_CDP (1u << 0)
>
> Is MBA have something like CDP as well, which can go through the
> same subop? Otherwise I think CAT would better remain present
> in this constant's name.
>
No. I will keep CAT for this macro.
> Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v4 03/15] x86: rename 'cbm_type' to 'psr_type' to make it general
2017-09-28 10:25 ` Jan Beulich
@ 2017-09-29 1:35 ` Yi Sun
0 siblings, 0 replies; 64+ messages in thread
From: Yi Sun @ 2017-09-29 1:35 UTC (permalink / raw)
To: Jan Beulich
Cc: Andrew Cooper, xen-devel, Wei Liu, Chao Peng, Roger Pau Monné
On 17-09-28 04:25:18, Jan Beulich wrote:
> >>> On 23.09.17 at 11:48, <yi.y.sun@linux.intel.com> wrote:
> > This patch renames 'cbm_type' to 'psr_type' to generalize it.
> > Then, we can reuse this for all psr allocation features.
> >
> > Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
> > Reviewed-by: Wei Liu <wei.liu2@citrix.com>
> > Reviewed-by: Roger Pau Monn? <roger.pau@citrix.com>
>
> Please could you try to get the name right here (so the committer
> won't have to fiddle with it); clearly your mailer has no problem
> with this, as can be seen ...
>
Oh, sorry for that. Will correct this.
> > ---
> > CC: Jan Beulich <jbeulich@suse.com>
> > CC: Andrew Cooper <andrew.cooper3@citrix.com>
> > CC: Wei Liu <wei.liu2@citrix.com>
> > CC: Roger Pau Monné <roger.pau@citrix.com>
>
> ... here.
>
> Acked-by: Jan Beulich <jbeulich@suse.com>
>
> Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v4 04/15] x86: implement data structure and CPU init flow for MBA
2017-09-28 11:00 ` Jan Beulich
@ 2017-09-29 1:53 ` Yi Sun
2017-10-04 5:52 ` Jan Beulich
0 siblings, 1 reply; 64+ messages in thread
From: Yi Sun @ 2017-09-29 1:53 UTC (permalink / raw)
To: Jan Beulich
Cc: Andrew Cooper, Roger Pau Monné, Wei Liu, Chao Peng, xen-devel
On 17-09-28 05:00:09, Jan Beulich wrote:
> >>> On 23.09.17 at 11:48, <yi.y.sun@linux.intel.com> wrote:
> > This patch implements main data structures of MBA.
> >
> > Like CAT features, MBA HW info has cos_max which means the max thrtl
> > register number, and thrtl_max which means the max throttle value
> > (delay value). It also has a flag to represent if the throttle
> > value is linear or not.
>
> Could you replace "or not" with what the alternative actually is,
> as "non-linear" can mean all sorts of things?
>
Sure.
> > One thrtl register of MBA stores a throttle value for one or more
> > domains. The throttle value means the delay between L2 cache and next
> > cache level.
>
> What is a delay between two cache levels?
>
There is a "programmable rate controller" between them to indirectly control
the bandwidth.
> > @@ -272,8 +293,8 @@ static bool psr_check_cbm(unsigned int cbm_len, unsigned long cbm)
> > return true;
> > }
> >
> > -/* CAT common functions implementation. */
> > -static int cat_init_feature(const struct cpuid_leaf *regs,
> > +/* Implementation of allocation features' functions. */
> > +static bool cat_init_feature(const struct cpuid_leaf *regs,
>
> Such a type change should happen in a separate patch, as this
> isn't specific to MBA. That way you can also make clear why you
> want this to change - the current description doesn't mention
> this at all.
>
Sure.
> > +static bool mba_init_feature(const struct cpuid_leaf *regs,
> > + struct feat_node *feat,
> > + struct psr_socket_info *info,
> > + enum psr_feat_type type)
> > +{
> > + /* No valid value so do not enable feature. */
> > + if ( !regs->a || !regs->d || type != FEAT_TYPE_MBA )
> > + return false;
> > +
> > + feat->cos_max = min(opt_cos_max, regs->d & CAT_COS_MAX_MASK);
> > + if ( feat->cos_max < 1 )
> > + return false;
> > +
> > + feat->mba.thrtl_max = (regs->a & MBA_THRTL_MAX_MASK) + 1;
> > +
> > + if ( regs->c & MBA_LINEAR_MASK )
> > + {
> > + feat->mba.linear = true;
> > +
> > + if ( feat->mba.thrtl_max >= 100 )
> > + return false;
> > + }
> > +
> > + wrmsrl(MSR_IA32_PSR_MBA_MASK(0), 0);
> > +
> > + /* Add this feature into array. */
> > + info->features[type] = feat;
> > +
> > + if ( !opt_cpu_info )
> > + return true;
> > +
> > + printk(XENLOG_INFO "MBA: enabled on socket %u, cos_max:%u, thrtl_max:%u, linear:%u.\n",
>
> The last one wants to be %d.
>
Ok, thanks!
> > @@ -1410,6 +1496,7 @@ static void psr_cpu_init(void)
> > unsigned int socket, cpu = smp_processor_id();
> > struct feat_node *feat;
> > struct cpuid_leaf regs;
> > + uint32_t ebx;
>
> Is this local variable really a big help? To me it looks like it only
> makes the patch larger without actually improving anything,
> and without being related to the subject of the patch.
>
IMHO, it can avoid the 'cpuid_count_leaf()' being repeatedly called. Without it,
we have to call 'cpuid_count_leaf()' for 2 more times.
I can move it to another patch to make it clear if you like it.
> Jan
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> https://lists.xen.org/xen-devel
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v4 07/15] x86: implement set value flow for MBA
2017-09-28 11:36 ` Jan Beulich
@ 2017-09-29 2:56 ` Yi Sun
2017-10-04 5:59 ` Jan Beulich
0 siblings, 1 reply; 64+ messages in thread
From: Yi Sun @ 2017-09-29 2:56 UTC (permalink / raw)
To: Jan Beulich
Cc: Andrew Cooper, xen-devel, Wei Liu, Chao Peng, Roger Pau Monné
On 17-09-28 05:36:11, Jan Beulich wrote:
> >>> On 23.09.17 at 11:48, <yi.y.sun@linux.intel.com> wrote:
> > This patch implements set value flow for MBA including its callback
> > function and domctl interface.
> >
> > It also changes the memebers in 'cos_write_info' to transfer the
> > feature array, feature properties array and value array. Then, we
> > can write all features values on the cos id into MSRs.
> >
> > Because multiple features may co-exist, we need handle all features to write
> > values of them into a COS register with new COS ID. E.g:
> > 1. L3 CAT and MBA co-exist.
> > 2. Dom1 and Dom2 share a same COS ID (2). The L3 CAT CBM of Dom1 is 0x1ff,
> > the MBA Thrtle of Dom1 is 0xa.
> > 3. User wants to change MBA Thrtl of Dom1 to be 0x14. Because COS ID 2 is
> > used by Dom2 too, we have to pick a new COS ID 3. The values of Dom1 on
> > COS ID 3 are all default values as below:
> > ---------
> > | COS 3 |
> > ---------
> > L3 CAT | 0x7ff |
> > ---------
> > MBA | 0x0 |
> > ---------
> > 4. After setting, the L3 CAT CBM value of Dom1 should be kept and the new MBA
> > Thrtl is set. So, the values on COS ID 3 should be below.
> > ---------
> > | COS 3 |
> > ---------
> > L3 CAT | 0x1ff |
> > ---------
> > MBA | 0x14 |
> > ---------
> >
> > So, we should write all features values into their MSRs. That requires the
> > feature array, feature properties array and value array are input.
>
> How is this last aspect (and the respective changes) related to MBA?
> I.e. why isn't this needed with the (also independent but possibly
> co-existing) L2/L3 CAT features?
>
I tried to introduce this in L2 CAT patch set but did not succeed. As there is
no HW that L2 CAT and L3 CAT co-exist so far, I did not insist on this.
> > --- a/xen/arch/x86/psr.c
> > +++ b/xen/arch/x86/psr.c
> > @@ -137,7 +137,10 @@ static const struct feat_props {
> > uint32_t data[], unsigned int array_len);
> >
> > /* write_msr is used to write out feature MSR register. */
> > - void (*write_msr)(unsigned int cos, uint32_t val, enum psr_type type);
> > + uint32_t (*write_msr)(unsigned int cos, uint32_t val, enum psr_type type);
>
> Again the type change would better be a prereq patch, to allow the
> focus here to be on MBA.
>
Sure, will move it to a new patch.
> > @@ -502,9 +514,23 @@ static bool mba_get_feat_info(const struct feat_node *feat,
> > return true;
> > }
> >
> > -static void mba_write_msr(unsigned int cos, uint32_t val,
> > - enum psr_type type)
> > +static uint32_t mba_write_msr(unsigned int cos, uint32_t val,
> > + enum psr_type type)
> > +{
> > + wrmsrl(MSR_IA32_PSR_MBA_MASK(cos), val);
> > +
> > + /* Read actual value set by hardware. */
> > + rdmsrl(MSR_IA32_PSR_MBA_MASK(cos), val);
> > +
> > + return val;
> > +}
> > +
> > +static bool mba_check_thrtl(const struct feat_node *feat, unsigned long thrtl)
> > {
> > + if ( thrtl > feat->mba.thrtl_max )
> > + return false;
> > +
> > + return true;
>
> This can be had with a single return statement.
>
Sure.
> > static void do_write_psr_msrs(void *data)
> > {
> > const struct cos_write_info *info = data;
> > - struct feat_node *feat = info->feature;
> > - const struct feat_props *props = info->props;
> > - unsigned int i, cos = info->cos, cos_num = props->cos_num;
> > + unsigned int i, index = 0, cos = info->cos;
> > + const uint32_t *val_array = info->val;
> >
> > - for ( i = 0; i < cos_num; i++ )
> > + for ( i = 0; i < ARRAY_SIZE(feat_props); i++ )
> > {
> > - if ( feat->cos_reg_val[cos * cos_num + i] != info->val[i] )
> > + struct feat_node *feat = info->features[i];
> > + const struct feat_props *props = info->props[i];
> > + unsigned int cos_num, j;
> > +
> > + if ( !feat || !props )
> > + continue;
> > +
> > + cos_num = props->cos_num;
> > + if ( info->array_len < index + cos_num )
> > + return;
> > +
> > + for ( j = 0; j < cos_num; j++ )
> > {
> > - feat->cos_reg_val[cos * cos_num + i] = info->val[i];
> > - props->write_msr(cos, info->val[i], props->type[i]);
> > + if ( feat->cos_reg_val[cos * cos_num + j] != val_array[index + j] )
> > + feat->cos_reg_val[cos * cos_num + j] =
> > + props->write_msr(cos, val_array[index + j], props->type[j]);
>
> This renders partly useless the check: If hardware can alter the
> value, repeatedly requesting the same value to be written will
> no longer guarantee the MSR write to be skipped. If hardware
> behavior can't be predicted you may want to consider recording
> both the value in found by reading back the register written and
> the value that was written - a match with either would eliminate
> the need to do the write.
>
The hardware behavior is explicitly defined by SDM and mentioned in
'xl-psr.markdown' and 'intel_psr_mba.pandoc'. User should know that HW
can alter MBA value if the value is not valid.
This check is not only for MBA but also for CAT features that the HW
cannot alter CAT value. Although this check is not a critical check,
it can prevent some non-necessary MSR write.
If you still think we should handle the case that user inputs an invalid
value every time, I think we can restore the codes in 'mba_check_thrtl'
to change invalid value to valid one, then insert the valid value into
val_array. Then, this check is always valid.
> Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v4 00/15] Enable Memory Bandwidth Allocation in Xen
2017-09-28 15:57 ` Wei Liu
@ 2017-09-29 2:58 ` Yi Sun
2017-09-29 8:02 ` Wei Liu
0 siblings, 1 reply; 64+ messages in thread
From: Yi Sun @ 2017-09-29 2:58 UTC (permalink / raw)
To: Wei Liu
Cc: Andrew Cooper, Ian Jackson, Julien Grall, Jan Beulich, Chao Peng,
xen-devel, Daniel De Graaf, Roger Pau Monn�
On 17-09-28 16:57:54, Wei Liu wrote:
> On Sat, Sep 23, 2017 at 05:48:09PM +0800, Yi Sun wrote:
> > Hi, all,
> >
> > We plan to bring a new PSR (Platform Shared Resource) feature called
> > Intel Memory Bandwidth Allocation (MBA) to Xen.
> >
> > Besides the MBA enabling, we change some interfaces to make them more
> > general but not only for CAT.
>
> I will get to this series next week. You already collected a bunch of
> comments, if you have addressed all of them, feel free to resend.
Thank you! Let me try to to address all comments before Chinese holiday.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v4 00/15] Enable Memory Bandwidth Allocation in Xen
2017-09-29 2:58 ` Yi Sun
@ 2017-09-29 8:02 ` Wei Liu
0 siblings, 0 replies; 64+ messages in thread
From: Wei Liu @ 2017-09-29 8:02 UTC (permalink / raw)
To: Yi Sun
Cc: Wei Liu, Andrew Cooper, Ian Jackson, Julien Grall, Jan Beulich,
Chao Peng, xen-devel, Daniel De Graaf, Roger Pau Monn�
On Fri, Sep 29, 2017 at 10:58:01AM +0800, Yi Sun wrote:
> On 17-09-28 16:57:54, Wei Liu wrote:
> > On Sat, Sep 23, 2017 at 05:48:09PM +0800, Yi Sun wrote:
> > > Hi, all,
> > >
> > > We plan to bring a new PSR (Platform Shared Resource) feature called
> > > Intel Memory Bandwidth Allocation (MBA) to Xen.
> > >
> > > Besides the MBA enabling, we change some interfaces to make them more
> > > general but not only for CAT.
> >
> > I will get to this series next week. You already collected a bunch of
> > comments, if you have addressed all of them, feel free to resend.
>
> Thank you! Let me try to to address all comments before Chinese holiday.
Enjoy your holiday!
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v4 02/15] Rename PSR sysctl/domctl interfaces and xsm policy to make them be general
2017-09-29 1:34 ` Yi Sun
@ 2017-10-04 5:47 ` Jan Beulich
0 siblings, 0 replies; 64+ messages in thread
From: Jan Beulich @ 2017-10-04 5:47 UTC (permalink / raw)
To: yi.y.sun
Cc: wei.liu2, andrew.cooper3, ian.jackson, xen-devel, chao.p.peng,
dgdegra, roger.pau
>>> Yi Sun <yi.y.sun@linux.intel.com> 09/29/17 3:37 AM >>>
>On 17-09-28 04:21:10, Jan Beulich wrote:
>> >>> On 23.09.17 at 11:48, <yi.y.sun@linux.intel.com> wrote:
>> > --- a/xen/include/public/domctl.h
>> > +++ b/xen/include/public/domctl.h
>> > @@ -37,7 +37,7 @@
>> > #include "hvm/save.h"
>> > #include "memory.h"
>> >
>> > -#define XEN_DOMCTL_INTERFACE_VERSION 0x0000000e
>> > +#define XEN_DOMCTL_INTERFACE_VERSION 0x0000000f
>>
>> This is needed only if the series doesn't make 4.10; 4.9 had it
>> ad 0x0000000d.
>>
>Thanks! So, this version number is upgraded only for one release. I thought
>every interface change should bump it.
It's for consumers to be in line with releases; breaking consumers between
unstable snapshots is not a concern (developers should be aware).
Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v4 04/15] x86: implement data structure and CPU init flow for MBA
2017-09-29 1:53 ` Yi Sun
@ 2017-10-04 5:52 ` Jan Beulich
2017-10-05 4:42 ` Yi Sun
0 siblings, 1 reply; 64+ messages in thread
From: Jan Beulich @ 2017-10-04 5:52 UTC (permalink / raw)
To: yi.y.sun; +Cc: andrew.cooper3, xen-devel, wei.liu2, chao.p.peng, roger.pau
>>> Yi Sun <yi.y.sun@linux.intel.com> 09/29/17 3:55 AM >>>
>On 17-09-28 05:00:09, Jan Beulich wrote:
>> >>> On 23.09.17 at 11:48, <yi.y.sun@linux.intel.com> wrote:
>> > One thrtl register of MBA stores a throttle value for one or more
>> > domains. The throttle value means the delay between L2 cache and next
>> > cache level.
>>
>> What is a delay between two cache levels?
>>
>There is a "programmable rate controller" between them to indirectly control
>the bandwidth.
I'm afraid I don't feel like this answers my question. Anyway - the sentence
needs some re-phrasing so it becomes clear what you're talking about.
>> > @@ -1410,6 +1496,7 @@ static void psr_cpu_init(void)
>> > unsigned int socket, cpu = smp_processor_id();
>> > struct feat_node *feat;
>> > struct cpuid_leaf regs;
>> > + uint32_t ebx;
>>
>> Is this local variable really a big help? To me it looks like it only
>> makes the patch larger without actually improving anything,
>> and without being related to the subject of the patch.
>>
>IMHO, it can avoid the 'cpuid_count_leaf()' being repeatedly called. Without it,
>we have to call 'cpuid_count_leaf()' for 2 more times.
Hmm, didn't you simply replace regs.b uses with ebx? Or did I overlook a place
where regs is being overwritten before the last of these regs.b uses (in which case
I think your change is fine)?
Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v4 07/15] x86: implement set value flow for MBA
2017-09-29 2:56 ` Yi Sun
@ 2017-10-04 5:59 ` Jan Beulich
2017-10-05 4:48 ` Yi Sun
0 siblings, 1 reply; 64+ messages in thread
From: Jan Beulich @ 2017-10-04 5:59 UTC (permalink / raw)
To: yi.y.sun; +Cc: andrew.cooper3, xen-devel, wei.liu2, chao.p.peng, roger.pau
>>> Yi Sun <yi.y.sun@linux.intel.com> 09/29/17 4:58 AM >>>
>On 17-09-28 05:36:11, Jan Beulich wrote:
>> >>> On 23.09.17 at 11:48, <yi.y.sun@linux.intel.com> wrote:
>> > This patch implements set value flow for MBA including its callback
>> > function and domctl interface.
>> >
>> > It also changes the memebers in 'cos_write_info' to transfer the
>> > feature array, feature properties array and value array. Then, we
>> > can write all features values on the cos id into MSRs.
>> >
>> > Because multiple features may co-exist, we need handle all features to write
>> > values of them into a COS register with new COS ID. E.g:
>> > 1. L3 CAT and MBA co-exist.
>> > 2. Dom1 and Dom2 share a same COS ID (2). The L3 CAT CBM of Dom1 is 0x1ff,
>> > the MBA Thrtle of Dom1 is 0xa.
>> > 3. User wants to change MBA Thrtl of Dom1 to be 0x14. Because COS ID 2 is
>> > used by Dom2 too, we have to pick a new COS ID 3. The values of Dom1 on
>> > COS ID 3 are all default values as below:
>> > ---------
>> > | COS 3 |
>> > ---------
>> > L3 CAT | 0x7ff |
>> > ---------
>> > MBA | 0x0 |
>> > ---------
>> > 4. After setting, the L3 CAT CBM value of Dom1 should be kept and the new MBA
>> > Thrtl is set. So, the values on COS ID 3 should be below.
>> > ---------
>> > | COS 3 |
>> > ---------
>> > L3 CAT | 0x1ff |
>> > ---------
>> > MBA | 0x14 |
>> > ---------
>> >
>> > So, we should write all features values into their MSRs. That requires the
>> > feature array, feature properties array and value array are input.
>>
>> How is this last aspect (and the respective changes) related to MBA?
>> I.e. why isn't this needed with the (also independent but possibly
>> co-existing) L2/L3 CAT features?
>>
>I tried to introduce this in L2 CAT patch set but did not succeed. As there is
>no HW that L2 CAT and L3 CAT co-exist so far, I did not insist on this.
Hmm, I'm afraid this wasn't then made clear enough to understand. I would
certainly not have been against something that could in theory occur with
L2/L3 CAT alone. In any event this means you don't want to mix this into this
MBA specific change here.
>> > static void do_write_psr_msrs(void *data)
>> > {
>> > const struct cos_write_info *info = data;
>> > - struct feat_node *feat = info->feature;
>> > - const struct feat_props *props = info->props;
>> > - unsigned int i, cos = info->cos, cos_num = props->cos_num;
>> > + unsigned int i, index = 0, cos = info->cos;
>> > + const uint32_t *val_array = info->val;
>> >
>> > - for ( i = 0; i < cos_num; i++ )
>> > + for ( i = 0; i < ARRAY_SIZE(feat_props); i++ )
>> > {
>> > - if ( feat->cos_reg_val[cos * cos_num + i] != info->val[i] )
>> > + struct feat_node *feat = info->features[i];
>> > + const struct feat_props *props = info->props[i];
>> > + unsigned int cos_num, j;
>> > +
>> > + if ( !feat || !props )
>> > + continue;
>> > +
>> > + cos_num = props->cos_num;
>> > + if ( info->array_len < index + cos_num )
>> > + return;
>> > +
>> > + for ( j = 0; j < cos_num; j++ )
>> > {
>> > - feat->cos_reg_val[cos * cos_num + i] = info->val[i];
>> > - props->write_msr(cos, info->val[i], props->type[i]);
>> > + if ( feat->cos_reg_val[cos * cos_num + j] != val_array[index + j] )
>> > + feat->cos_reg_val[cos * cos_num + j] =
>> > + props->write_msr(cos, val_array[index + j], props->type[j]);
>>
>> This renders partly useless the check: If hardware can alter the
>> value, repeatedly requesting the same value to be written will
>> no longer guarantee the MSR write to be skipped. If hardware
>> behavior can't be predicted you may want to consider recording
>> both the value in found by reading back the register written and
>> the value that was written - a match with either would eliminate
>> the need to do the write.
>>
>The hardware behavior is explicitly defined by SDM and mentioned in
>'xl-psr.markdown' and 'intel_psr_mba.pandoc'. User should know that HW
>can alter MBA value if the value is not valid.
So if hardware behavior is fully defined, why don't you pre-adjust what is
to be written to the value hardware would alter it to?
>This check is not only for MBA but also for CAT features that the HW
>cannot alter CAT value.
I don't understand this part.
> Although this check is not a critical check,
>it can prevent some non-necessary MSR write.
That's my point - while previously all unnecessary writes were avoided,
you now avoid only some.
>If you still think we should handle the case that user inputs an invalid
>value every time, I think we can restore the codes in 'mba_check_thrtl'
>to change invalid value to valid one, then insert the valid value into
>val_array. Then, this check is always valid.
I don't think I've asked to deal with "invalid" values here (which should be
rejected anyway, but that's a different topic). Values adjusted by hardware
don't fall into the "invalid" category for me.
Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v4 04/15] x86: implement data structure and CPU init flow for MBA
2017-10-04 5:52 ` Jan Beulich
@ 2017-10-05 4:42 ` Yi Sun
2017-10-05 8:49 ` Jan Beulich
0 siblings, 1 reply; 64+ messages in thread
From: Yi Sun @ 2017-10-05 4:42 UTC (permalink / raw)
To: Jan Beulich; +Cc: andrew.cooper3, xen-devel, wei.liu2, chao.p.peng, roger.pau
On 17-10-03 23:52:09, Jan Beulich wrote:
> >>> Yi Sun <yi.y.sun@linux.intel.com> 09/29/17 3:55 AM >>>
> >On 17-09-28 05:00:09, Jan Beulich wrote:
> >> >>> On 23.09.17 at 11:48, <yi.y.sun@linux.intel.com> wrote:
> >> > One thrtl register of MBA stores a throttle value for one or more
> >> > domains. The throttle value means the delay between L2 cache and next
> >> > cache level.
> >>
> >> What is a delay between two cache levels?
> >>
> >There is a "programmable rate controller" between them to indirectly control
> >the bandwidth.
>
> I'm afraid I don't feel like this answers my question. Anyway - the sentence
> needs some re-phrasing so it becomes clear what you're talking about.
>
I may draw the figure shown in SDM to make things clear.
> >> > @@ -1410,6 +1496,7 @@ static void psr_cpu_init(void)
> >> > unsigned int socket, cpu = smp_processor_id();
> >> > struct feat_node *feat;
> >> > struct cpuid_leaf regs;
> >> > + uint32_t ebx;
> >>
> >> Is this local variable really a big help? To me it looks like it only
> >> makes the patch larger without actually improving anything,
> >> and without being related to the subject of the patch.
> >>
> >IMHO, it can avoid the 'cpuid_count_leaf()' being repeatedly called. Without it,
> >we have to call 'cpuid_count_leaf()' for 2 more times.
>
> Hmm, didn't you simply replace regs.b uses with ebx? Or did I overlook a place
> where regs is being overwritten before the last of these regs.b uses (in which case
> I think your change is fine)?
>
The regs is overwritten when a feature presents. The old codes are below
cpuid_count_leaf(PSR_CPUID_LEVEL_CAT, 0, ®s);
if ( regs.b & PSR_RESOURCE_TYPE_L3 )
{
cpuid_count_leaf(PSR_CPUID_LEVEL_CAT, 1, ®s); //It is overwritten here.
......
}
cpuid_count_leaf(PSR_CPUID_LEVEL_CAT, 0, ®s); //So, we have to call cpuid to get regs again.
if ( regs.b & PSR_RESOURCE_TYPE_L2 )
{
cpuid_count_leaf(PSR_CPUID_LEVEL_CAT, 2, ®s);
......
Because above reason, I defined this ebx local variable to avoid calling cpuid
again for next feature.
> Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v4 07/15] x86: implement set value flow for MBA
2017-10-04 5:59 ` Jan Beulich
@ 2017-10-05 4:48 ` Yi Sun
2017-10-05 8:39 ` Roger Pau Monné
2017-10-05 8:55 ` Jan Beulich
0 siblings, 2 replies; 64+ messages in thread
From: Yi Sun @ 2017-10-05 4:48 UTC (permalink / raw)
To: Jan Beulich; +Cc: andrew.cooper3, xen-devel, wei.liu2, chao.p.peng, roger.pau
On 17-10-03 23:59:46, Jan Beulich wrote:
> >>> Yi Sun <yi.y.sun@linux.intel.com> 09/29/17 4:58 AM >>>
> >On 17-09-28 05:36:11, Jan Beulich wrote:
> >> >>> On 23.09.17 at 11:48, <yi.y.sun@linux.intel.com> wrote:
> >> > This patch implements set value flow for MBA including its callback
> >> > function and domctl interface.
> >> >
> >> > It also changes the memebers in 'cos_write_info' to transfer the
> >> > feature array, feature properties array and value array. Then, we
> >> > can write all features values on the cos id into MSRs.
> >> >
> >> > Because multiple features may co-exist, we need handle all features to write
> >> > values of them into a COS register with new COS ID. E.g:
> >> > 1. L3 CAT and MBA co-exist.
> >> > 2. Dom1 and Dom2 share a same COS ID (2). The L3 CAT CBM of Dom1 is 0x1ff,
> >> > the MBA Thrtle of Dom1 is 0xa.
> >> > 3. User wants to change MBA Thrtl of Dom1 to be 0x14. Because COS ID 2 is
> >> > used by Dom2 too, we have to pick a new COS ID 3. The values of Dom1 on
> >> > COS ID 3 are all default values as below:
> >> > ---------
> >> > | COS 3 |
> >> > ---------
> >> > L3 CAT | 0x7ff |
> >> > ---------
> >> > MBA | 0x0 |
> >> > ---------
> >> > 4. After setting, the L3 CAT CBM value of Dom1 should be kept and the new MBA
> >> > Thrtl is set. So, the values on COS ID 3 should be below.
> >> > ---------
> >> > | COS 3 |
> >> > ---------
> >> > L3 CAT | 0x1ff |
> >> > ---------
> >> > MBA | 0x14 |
> >> > ---------
> >> >
> >> > So, we should write all features values into their MSRs. That requires the
> >> > feature array, feature properties array and value array are input.
> >>
> >> How is this last aspect (and the respective changes) related to MBA?
> >> I.e. why isn't this needed with the (also independent but possibly
> >> co-existing) L2/L3 CAT features?
> >>
> >I tried to introduce this in L2 CAT patch set but did not succeed. As there is
> >no HW that L2 CAT and L3 CAT co-exist so far, I did not insist on this.
>
> Hmm, I'm afraid this wasn't then made clear enough to understand. I would
> certainly not have been against something that could in theory occur with
> L2/L3 CAT alone. In any event this means you don't want to mix this into this
> MBA specific change here.
>
Anyway, I think you suggest to split this as a new patch, right?
> >> > static void do_write_psr_msrs(void *data)
> >> > {
> >> > const struct cos_write_info *info = data;
> >> > - struct feat_node *feat = info->feature;
> >> > - const struct feat_props *props = info->props;
> >> > - unsigned int i, cos = info->cos, cos_num = props->cos_num;
> >> > + unsigned int i, index = 0, cos = info->cos;
> >> > + const uint32_t *val_array = info->val;
> >> >
> >> > - for ( i = 0; i < cos_num; i++ )
> >> > + for ( i = 0; i < ARRAY_SIZE(feat_props); i++ )
> >> > {
> >> > - if ( feat->cos_reg_val[cos * cos_num + i] != info->val[i] )
> >> > + struct feat_node *feat = info->features[i];
> >> > + const struct feat_props *props = info->props[i];
> >> > + unsigned int cos_num, j;
> >> > +
> >> > + if ( !feat || !props )
> >> > + continue;
> >> > +
> >> > + cos_num = props->cos_num;
> >> > + if ( info->array_len < index + cos_num )
> >> > + return;
> >> > +
> >> > + for ( j = 0; j < cos_num; j++ )
> >> > {
> >> > - feat->cos_reg_val[cos * cos_num + i] = info->val[i];
> >> > - props->write_msr(cos, info->val[i], props->type[i]);
> >> > + if ( feat->cos_reg_val[cos * cos_num + j] != val_array[index + j] )
> >> > + feat->cos_reg_val[cos * cos_num + j] =
> >> > + props->write_msr(cos, val_array[index + j], props->type[j]);
> >>
> >> This renders partly useless the check: If hardware can alter the
> >> value, repeatedly requesting the same value to be written will
> >> no longer guarantee the MSR write to be skipped. If hardware
> >> behavior can't be predicted you may want to consider recording
> >> both the value in found by reading back the register written and
> >> the value that was written - a match with either would eliminate
> >> the need to do the write.
> >>
> >The hardware behavior is explicitly defined by SDM and mentioned in
> >'xl-psr.markdown' and 'intel_psr_mba.pandoc'. User should know that HW
> >can alter MBA value if the value is not valid.
>
> So if hardware behavior is fully defined, why don't you pre-adjust what is
> to be written to the value hardware would alter it to?
>
In previous version of MBA patch set, I pre-adjust the value in 'mba_check_thrtl'.
But Roger did not like that. So, the pre-adjust codes are removed.
> >This check is not only for MBA but also for CAT features that the HW
> >cannot alter CAT value.
>
> I don't understand this part.
>
I mean the check here are for all features so we cannot remove it.
> > Although this check is not a critical check,
> >it can prevent some non-necessary MSR write.
>
> That's my point - while previously all unnecessary writes were avoided,
> you now avoid only some.
>
Without the pre-adjust codes in 'mba_check_thrtl', if user inputs value, e.g.
11/22/33/..., this check cannot prevent the write action. So, only some can
be avoided in current codes.
> >If you still think we should handle the case that user inputs an invalid
> >value every time, I think we can restore the codes in 'mba_check_thrtl'
> >to change invalid value to valid one, then insert the valid value into
> >val_array. Then, this check is always valid.
>
> I don't think I've asked to deal with "invalid" values here (which should be
> rejected anyway, but that's a different topic). Values adjusted by hardware
> don't fall into the "invalid" category for me.
>
If the pre-adjust codes in 'mba_check_thrtl' are restored, all values written
to HW are valid.
> Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v4 07/15] x86: implement set value flow for MBA
2017-10-05 4:48 ` Yi Sun
@ 2017-10-05 8:39 ` Roger Pau Monné
2017-10-05 9:39 ` Jan Beulich
2017-10-05 8:55 ` Jan Beulich
1 sibling, 1 reply; 64+ messages in thread
From: Roger Pau Monné @ 2017-10-05 8:39 UTC (permalink / raw)
To: Yi Sun; +Cc: andrew.cooper3, wei.liu2, xen-devel, Jan Beulich, chao.p.peng
On Thu, Oct 05, 2017 at 04:48:12AM +0000, Yi Sun wrote:
> On 17-10-03 23:59:46, Jan Beulich wrote:
> > >>> Yi Sun <yi.y.sun@linux.intel.com> 09/29/17 4:58 AM >>>
> > >On 17-09-28 05:36:11, Jan Beulich wrote:
> > >> >>> On 23.09.17 at 11:48, <yi.y.sun@linux.intel.com> wrote:
> > >> > - feat->cos_reg_val[cos * cos_num + i] = info->val[i];
> > >> > - props->write_msr(cos, info->val[i], props->type[i]);
> > >> > + if ( feat->cos_reg_val[cos * cos_num + j] != val_array[index + j] )
> > >> > + feat->cos_reg_val[cos * cos_num + j] =
> > >> > + props->write_msr(cos, val_array[index + j], props->type[j]);
> > >>
> > >> This renders partly useless the check: If hardware can alter the
> > >> value, repeatedly requesting the same value to be written will
> > >> no longer guarantee the MSR write to be skipped. If hardware
> > >> behavior can't be predicted you may want to consider recording
> > >> both the value in found by reading back the register written and
> > >> the value that was written - a match with either would eliminate
> > >> the need to do the write.
> > >>
> > >The hardware behavior is explicitly defined by SDM and mentioned in
> > >'xl-psr.markdown' and 'intel_psr_mba.pandoc'. User should know that HW
> > >can alter MBA value if the value is not valid.
> >
> > So if hardware behavior is fully defined, why don't you pre-adjust what is
> > to be written to the value hardware would alter it to?
> >
> In previous version of MBA patch set, I pre-adjust the value in 'mba_check_thrtl'.
> But Roger did not like that. So, the pre-adjust codes are removed.
IMHO it's quite pointless to do such adjustments when the hardware
performs them already. Also, I fear that our adjustments might get
out-of-sync in the future with what hardware actually does.
Maybe the result read back from the hardware (ie: adjusted) can be
stored and used in order to check whether a new value should be
written or not when switching? (I think this is the same that Jan
suggested above).
Roger.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v4 04/15] x86: implement data structure and CPU init flow for MBA
2017-10-05 4:42 ` Yi Sun
@ 2017-10-05 8:49 ` Jan Beulich
2017-10-05 10:08 ` Yi Sun
0 siblings, 1 reply; 64+ messages in thread
From: Jan Beulich @ 2017-10-05 8:49 UTC (permalink / raw)
To: Yi Sun; +Cc: andrew.cooper3, xen-devel, wei.liu2, chao.p.peng, roger.pau
>>> On 05.10.17 at 06:42, <yi.y.sun@linux.intel.com> wrote:
> On 17-10-03 23:52:09, Jan Beulich wrote:
>> >>> Yi Sun <yi.y.sun@linux.intel.com> 09/29/17 3:55 AM >>>
>> >On 17-09-28 05:00:09, Jan Beulich wrote:
>> >> >>> On 23.09.17 at 11:48, <yi.y.sun@linux.intel.com> wrote:
>> >> > @@ -1410,6 +1496,7 @@ static void psr_cpu_init(void)
>> >> > unsigned int socket, cpu = smp_processor_id();
>> >> > struct feat_node *feat;
>> >> > struct cpuid_leaf regs;
>> >> > + uint32_t ebx;
>> >>
>> >> Is this local variable really a big help? To me it looks like it only
>> >> makes the patch larger without actually improving anything,
>> >> and without being related to the subject of the patch.
>> >>
>> >IMHO, it can avoid the 'cpuid_count_leaf()' being repeatedly called. Without it,
>> >we have to call 'cpuid_count_leaf()' for 2 more times.
>>
>> Hmm, didn't you simply replace regs.b uses with ebx? Or did I overlook a place
>> where regs is being overwritten before the last of these regs.b uses (in which case
>> I think your change is fine)?
>>
> The regs is overwritten when a feature presents. The old codes are below
>
> cpuid_count_leaf(PSR_CPUID_LEVEL_CAT, 0, ®s);
> if ( regs.b & PSR_RESOURCE_TYPE_L3 )
> {
> cpuid_count_leaf(PSR_CPUID_LEVEL_CAT, 1, ®s); //It is overwritten here.
> ......
> }
>
> cpuid_count_leaf(PSR_CPUID_LEVEL_CAT, 0, ®s); //So, we have to call cpuid to get regs again.
> if ( regs.b & PSR_RESOURCE_TYPE_L2 )
> {
> cpuid_count_leaf(PSR_CPUID_LEVEL_CAT, 2, ®s);
> ......
>
> Because above reason, I defined this ebx local variable to avoid calling cpuid
> again for next feature.
I see. But then please give the variable a better name, reflecting
the data it holds.
Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v4 07/15] x86: implement set value flow for MBA
2017-10-05 4:48 ` Yi Sun
2017-10-05 8:39 ` Roger Pau Monné
@ 2017-10-05 8:55 ` Jan Beulich
2017-10-05 10:32 ` Yi Sun
1 sibling, 1 reply; 64+ messages in thread
From: Jan Beulich @ 2017-10-05 8:55 UTC (permalink / raw)
To: Yi Sun; +Cc: andrew.cooper3, xen-devel, wei.liu2, chao.p.peng, roger.pau
>>> On 05.10.17 at 06:48, <yi.y.sun@linux.intel.com> wrote:
> On 17-10-03 23:59:46, Jan Beulich wrote:
>> >>> Yi Sun <yi.y.sun@linux.intel.com> 09/29/17 4:58 AM >>>
>> >On 17-09-28 05:36:11, Jan Beulich wrote:
>> >> >>> On 23.09.17 at 11:48, <yi.y.sun@linux.intel.com> wrote:
>> >> > This patch implements set value flow for MBA including its callback
>> >> > function and domctl interface.
>> >> >
>> >> > It also changes the memebers in 'cos_write_info' to transfer the
>> >> > feature array, feature properties array and value array. Then, we
>> >> > can write all features values on the cos id into MSRs.
>> >> >
>> >> > Because multiple features may co-exist, we need handle all features to write
>> >> > values of them into a COS register with new COS ID. E.g:
>> >> > 1. L3 CAT and MBA co-exist.
>> >> > 2. Dom1 and Dom2 share a same COS ID (2). The L3 CAT CBM of Dom1 is 0x1ff,
>> >> > the MBA Thrtle of Dom1 is 0xa.
>> >> > 3. User wants to change MBA Thrtl of Dom1 to be 0x14. Because COS ID 2 is
>> >> > used by Dom2 too, we have to pick a new COS ID 3. The values of Dom1 on
>> >> > COS ID 3 are all default values as below:
>> >> > ---------
>> >> > | COS 3 |
>> >> > ---------
>> >> > L3 CAT | 0x7ff |
>> >> > ---------
>> >> > MBA | 0x0 |
>> >> > ---------
>> >> > 4. After setting, the L3 CAT CBM value of Dom1 should be kept and the new MBA
>> >> > Thrtl is set. So, the values on COS ID 3 should be below.
>> >> > ---------
>> >> > | COS 3 |
>> >> > ---------
>> >> > L3 CAT | 0x1ff |
>> >> > ---------
>> >> > MBA | 0x14 |
>> >> > ---------
>> >> >
>> >> > So, we should write all features values into their MSRs. That requires the
>> >> > feature array, feature properties array and value array are input.
>> >>
>> >> How is this last aspect (and the respective changes) related to MBA?
>> >> I.e. why isn't this needed with the (also independent but possibly
>> >> co-existing) L2/L3 CAT features?
>> >>
>> >I tried to introduce this in L2 CAT patch set but did not succeed. As there is
>> >no HW that L2 CAT and L3 CAT co-exist so far, I did not insist on this.
>>
>> Hmm, I'm afraid this wasn't then made clear enough to understand. I would
>> certainly not have been against something that could in theory occur with
>> L2/L3 CAT alone. In any event this means you don't want to mix this into this
>> MBA specific change here.
>>
> Anyway, I think you suggest to split this as a new patch, right?
Yes indeed.
>> >> > static void do_write_psr_msrs(void *data)
>> >> > {
>> >> > const struct cos_write_info *info = data;
>> >> > - struct feat_node *feat = info->feature;
>> >> > - const struct feat_props *props = info->props;
>> >> > - unsigned int i, cos = info->cos, cos_num = props->cos_num;
>> >> > + unsigned int i, index = 0, cos = info->cos;
>> >> > + const uint32_t *val_array = info->val;
>> >> >
>> >> > - for ( i = 0; i < cos_num; i++ )
>> >> > + for ( i = 0; i < ARRAY_SIZE(feat_props); i++ )
>> >> > {
>> >> > - if ( feat->cos_reg_val[cos * cos_num + i] != info->val[i] )
>> >> > + struct feat_node *feat = info->features[i];
>> >> > + const struct feat_props *props = info->props[i];
>> >> > + unsigned int cos_num, j;
>> >> > +
>> >> > + if ( !feat || !props )
>> >> > + continue;
>> >> > +
>> >> > + cos_num = props->cos_num;
>> >> > + if ( info->array_len < index + cos_num )
>> >> > + return;
>> >> > +
>> >> > + for ( j = 0; j < cos_num; j++ )
>> >> > {
>> >> > - feat->cos_reg_val[cos * cos_num + i] = info->val[i];
>> >> > - props->write_msr(cos, info->val[i], props->type[i]);
>> >> > + if ( feat->cos_reg_val[cos * cos_num + j] != val_array[index + j] )
>> >> > + feat->cos_reg_val[cos * cos_num + j] =
>> >> > + props->write_msr(cos, val_array[index + j], props->type[j]);
>> >>
>> >> This renders partly useless the check: If hardware can alter the
>> >> value, repeatedly requesting the same value to be written will
>> >> no longer guarantee the MSR write to be skipped. If hardware
>> >> behavior can't be predicted you may want to consider recording
>> >> both the value in found by reading back the register written and
>> >> the value that was written - a match with either would eliminate
>> >> the need to do the write.
>> >>
>> >The hardware behavior is explicitly defined by SDM and mentioned in
>> >'xl-psr.markdown' and 'intel_psr_mba.pandoc'. User should know that HW
>> >can alter MBA value if the value is not valid.
>>
>> So if hardware behavior is fully defined, why don't you pre-adjust what is
>> to be written to the value hardware would alter it to?
>>
> In previous version of MBA patch set, I pre-adjust the value in 'mba_check_thrtl'.
> But Roger did not like that. So, the pre-adjust codes are removed.
Could you point me at or repeat the reason(s) of his dislike?
>> >This check is not only for MBA but also for CAT features that the HW
>> >cannot alter CAT value.
>>
>> I don't understand this part.
>>
> I mean the check here are for all features so we cannot remove it.
I _still_ don't understand: If the check can't be removed (even
without MBA in mind), then the implication would be that the
code prior to this series is buggy. In which case I'd expect you to
submit a standalone bug fix, rather than mixing the fix into here.
>> > Although this check is not a critical check,
>> >it can prevent some non-necessary MSR write.
>>
>> That's my point - while previously all unnecessary writes were avoided,
>> you now avoid only some.
>>
> Without the pre-adjust codes in 'mba_check_thrtl', if user inputs value, e.g.
> 11/22/33/..., this check cannot prevent the write action. So, only some can
> be avoided in current codes.
Right. If it's worthwhile avoiding the writes, all of them should be
avoided when the resulting value isn't going to change. Otherwise
the write avoidance logic can/should be dropped altogether.
Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v4 07/15] x86: implement set value flow for MBA
2017-10-05 8:39 ` Roger Pau Monné
@ 2017-10-05 9:39 ` Jan Beulich
2017-10-05 10:10 ` Yi Sun
0 siblings, 1 reply; 64+ messages in thread
From: Jan Beulich @ 2017-10-05 9:39 UTC (permalink / raw)
To: Roger Pau Monné, Yi Sun
Cc: andrew.cooper3, wei.liu2, xen-devel, chao.p.peng
>>> On 05.10.17 at 10:39, <roger.pau@citrix.com> wrote:
> On Thu, Oct 05, 2017 at 04:48:12AM +0000, Yi Sun wrote:
>> On 17-10-03 23:59:46, Jan Beulich wrote:
>> > >>> Yi Sun <yi.y.sun@linux.intel.com> 09/29/17 4:58 AM >>>
>> > >On 17-09-28 05:36:11, Jan Beulich wrote:
>> > >> >>> On 23.09.17 at 11:48, <yi.y.sun@linux.intel.com> wrote:
>> > >> > - feat->cos_reg_val[cos * cos_num + i] = info->val[i];
>> > >> > - props->write_msr(cos, info->val[i], props->type[i]);
>> > >> > + if ( feat->cos_reg_val[cos * cos_num + j] != val_array[index +
> j] )
>> > >> > + feat->cos_reg_val[cos * cos_num + j] =
>> > >> > + props->write_msr(cos, val_array[index + j],
> props->type[j]);
>> > >>
>> > >> This renders partly useless the check: If hardware can alter the
>> > >> value, repeatedly requesting the same value to be written will
>> > >> no longer guarantee the MSR write to be skipped. If hardware
>> > >> behavior can't be predicted you may want to consider recording
>> > >> both the value in found by reading back the register written and
>> > >> the value that was written - a match with either would eliminate
>> > >> the need to do the write.
>> > >>
>> > >The hardware behavior is explicitly defined by SDM and mentioned in
>> > >'xl-psr.markdown' and 'intel_psr_mba.pandoc'. User should know that HW
>> > >can alter MBA value if the value is not valid.
>> >
>> > So if hardware behavior is fully defined, why don't you pre-adjust what is
>> > to be written to the value hardware would alter it to?
>> >
>> In previous version of MBA patch set, I pre-adjust the value in
> 'mba_check_thrtl'.
>> But Roger did not like that. So, the pre-adjust codes are removed.
>
> IMHO it's quite pointless to do such adjustments when the hardware
> performs them already. Also, I fear that our adjustments might get
> out-of-sync in the future with what hardware actually does.
>
> Maybe the result read back from the hardware (ie: adjusted) can be
> stored and used in order to check whether a new value should be
> written or not when switching? (I think this is the same that Jan
> suggested above).
Not exactly, no - I'd like to avoid the write for _any_ value
resulting in the one currently stored in the hardware register.
Hence my earlier question on whether the transformation
done by hardware is well defined (i.e. _not_ model dependent
or fully defined by CPUID output).
Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v4 04/15] x86: implement data structure and CPU init flow for MBA
2017-10-05 8:49 ` Jan Beulich
@ 2017-10-05 10:08 ` Yi Sun
0 siblings, 0 replies; 64+ messages in thread
From: Yi Sun @ 2017-10-05 10:08 UTC (permalink / raw)
To: Jan Beulich; +Cc: andrew.cooper3, xen-devel, wei.liu2, chao.p.peng, roger.pau
On 17-10-05 02:49:59, Jan Beulich wrote:
> >>> On 05.10.17 at 06:42, <yi.y.sun@linux.intel.com> wrote:
> > On 17-10-03 23:52:09, Jan Beulich wrote:
> >> >>> Yi Sun <yi.y.sun@linux.intel.com> 09/29/17 3:55 AM >>>
> >> >On 17-09-28 05:00:09, Jan Beulich wrote:
> >> >> >>> On 23.09.17 at 11:48, <yi.y.sun@linux.intel.com> wrote:
> >> >> > @@ -1410,6 +1496,7 @@ static void psr_cpu_init(void)
> >> >> > unsigned int socket, cpu = smp_processor_id();
> >> >> > struct feat_node *feat;
> >> >> > struct cpuid_leaf regs;
> >> >> > + uint32_t ebx;
> >> >>
> >> >> Is this local variable really a big help? To me it looks like it only
> >> >> makes the patch larger without actually improving anything,
> >> >> and without being related to the subject of the patch.
> >> >>
> >> >IMHO, it can avoid the 'cpuid_count_leaf()' being repeatedly called. Without it,
> >> >we have to call 'cpuid_count_leaf()' for 2 more times.
> >>
> >> Hmm, didn't you simply replace regs.b uses with ebx? Or did I overlook a place
> >> where regs is being overwritten before the last of these regs.b uses (in which case
> >> I think your change is fine)?
> >>
> > The regs is overwritten when a feature presents. The old codes are below
> >
> > cpuid_count_leaf(PSR_CPUID_LEVEL_CAT, 0, ®s);
> > if ( regs.b & PSR_RESOURCE_TYPE_L3 )
> > {
> > cpuid_count_leaf(PSR_CPUID_LEVEL_CAT, 1, ®s); //It is overwritten here.
> > ......
> > }
> >
> > cpuid_count_leaf(PSR_CPUID_LEVEL_CAT, 0, ®s); //So, we have to call cpuid to get regs again.
> > if ( regs.b & PSR_RESOURCE_TYPE_L2 )
> > {
> > cpuid_count_leaf(PSR_CPUID_LEVEL_CAT, 2, ®s);
> > ......
> >
> > Because above reason, I defined this ebx local variable to avoid calling cpuid
> > again for next feature.
>
> I see. But then please give the variable a better name, reflecting
> the data it holds.
>
Then, how about 'feat_mask'?
> Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v4 07/15] x86: implement set value flow for MBA
2017-10-05 9:39 ` Jan Beulich
@ 2017-10-05 10:10 ` Yi Sun
0 siblings, 0 replies; 64+ messages in thread
From: Yi Sun @ 2017-10-05 10:10 UTC (permalink / raw)
To: Jan Beulich
Cc: andrew.cooper3, xen-devel, wei.liu2, chao.p.peng, Roger Pau Monné
On 17-10-05 03:39:06, Jan Beulich wrote:
> >>> On 05.10.17 at 10:39, <roger.pau@citrix.com> wrote:
> > On Thu, Oct 05, 2017 at 04:48:12AM +0000, Yi Sun wrote:
> >> On 17-10-03 23:59:46, Jan Beulich wrote:
> >> > >>> Yi Sun <yi.y.sun@linux.intel.com> 09/29/17 4:58 AM >>>
> >> > >On 17-09-28 05:36:11, Jan Beulich wrote:
> >> > >> >>> On 23.09.17 at 11:48, <yi.y.sun@linux.intel.com> wrote:
> >> > >> > - feat->cos_reg_val[cos * cos_num + i] = info->val[i];
> >> > >> > - props->write_msr(cos, info->val[i], props->type[i]);
> >> > >> > + if ( feat->cos_reg_val[cos * cos_num + j] != val_array[index +
> > j] )
> >> > >> > + feat->cos_reg_val[cos * cos_num + j] =
> >> > >> > + props->write_msr(cos, val_array[index + j],
> > props->type[j]);
> >> > >>
> >> > >> This renders partly useless the check: If hardware can alter the
> >> > >> value, repeatedly requesting the same value to be written will
> >> > >> no longer guarantee the MSR write to be skipped. If hardware
> >> > >> behavior can't be predicted you may want to consider recording
> >> > >> both the value in found by reading back the register written and
> >> > >> the value that was written - a match with either would eliminate
> >> > >> the need to do the write.
> >> > >>
> >> > >The hardware behavior is explicitly defined by SDM and mentioned in
> >> > >'xl-psr.markdown' and 'intel_psr_mba.pandoc'. User should know that HW
> >> > >can alter MBA value if the value is not valid.
> >> >
> >> > So if hardware behavior is fully defined, why don't you pre-adjust what is
> >> > to be written to the value hardware would alter it to?
> >> >
> >> In previous version of MBA patch set, I pre-adjust the value in
> > 'mba_check_thrtl'.
> >> But Roger did not like that. So, the pre-adjust codes are removed.
> >
> > IMHO it's quite pointless to do such adjustments when the hardware
> > performs them already. Also, I fear that our adjustments might get
> > out-of-sync in the future with what hardware actually does.
> >
> > Maybe the result read back from the hardware (ie: adjusted) can be
> > stored and used in order to check whether a new value should be
> > written or not when switching? (I think this is the same that Jan
> > suggested above).
>
> Not exactly, no - I'd like to avoid the write for _any_ value
> resulting in the one currently stored in the hardware register.
> Hence my earlier question on whether the transformation
> done by hardware is well defined (i.e. _not_ model dependent
> or fully defined by CPUID output).
>
SDM does not mention it is model dependent. So, it is _not_ model dependent.
> Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v4 07/15] x86: implement set value flow for MBA
2017-10-05 8:55 ` Jan Beulich
@ 2017-10-05 10:32 ` Yi Sun
0 siblings, 0 replies; 64+ messages in thread
From: Yi Sun @ 2017-10-05 10:32 UTC (permalink / raw)
To: Jan Beulich; +Cc: andrew.cooper3, xen-devel, wei.liu2, chao.p.peng, roger.pau
On 17-10-05 02:55:17, Jan Beulich wrote:
> >>> On 05.10.17 at 06:48, <yi.y.sun@linux.intel.com> wrote:
> > On 17-10-03 23:59:46, Jan Beulich wrote:
> >> >>> Yi Sun <yi.y.sun@linux.intel.com> 09/29/17 4:58 AM >>>
> >> >On 17-09-28 05:36:11, Jan Beulich wrote:
> >> >> >>> On 23.09.17 at 11:48, <yi.y.sun@linux.intel.com> wrote:
[...]
> >> >> > {
> >> >> > const struct cos_write_info *info = data;
> >> >> > - struct feat_node *feat = info->feature;
> >> >> > - const struct feat_props *props = info->props;
> >> >> > - unsigned int i, cos = info->cos, cos_num = props->cos_num;
> >> >> > + unsigned int i, index = 0, cos = info->cos;
> >> >> > + const uint32_t *val_array = info->val;
> >> >> >
> >> >> > - for ( i = 0; i < cos_num; i++ )
> >> >> > + for ( i = 0; i < ARRAY_SIZE(feat_props); i++ )
> >> >> > {
> >> >> > - if ( feat->cos_reg_val[cos * cos_num + i] != info->val[i] )
> >> >> > + struct feat_node *feat = info->features[i];
> >> >> > + const struct feat_props *props = info->props[i];
> >> >> > + unsigned int cos_num, j;
> >> >> > +
> >> >> > + if ( !feat || !props )
> >> >> > + continue;
> >> >> > +
> >> >> > + cos_num = props->cos_num;
> >> >> > + if ( info->array_len < index + cos_num )
> >> >> > + return;
> >> >> > +
> >> >> > + for ( j = 0; j < cos_num; j++ )
> >> >> > {
> >> >> > - feat->cos_reg_val[cos * cos_num + i] = info->val[i];
> >> >> > - props->write_msr(cos, info->val[i], props->type[i]);
> >> >> > + if ( feat->cos_reg_val[cos * cos_num + j] != val_array[index + j] )
> >> >> > + feat->cos_reg_val[cos * cos_num + j] =
> >> >> > + props->write_msr(cos, val_array[index + j], props->type[j]);
> >> >>
> >> >> This renders partly useless the check: If hardware can alter the
> >> >> value, repeatedly requesting the same value to be written will
> >> >> no longer guarantee the MSR write to be skipped. If hardware
> >> >> behavior can't be predicted you may want to consider recording
> >> >> both the value in found by reading back the register written and
> >> >> the value that was written - a match with either would eliminate
> >> >> the need to do the write.
> >> >>
> >> >The hardware behavior is explicitly defined by SDM and mentioned in
> >> >'xl-psr.markdown' and 'intel_psr_mba.pandoc'. User should know that HW
> >> >can alter MBA value if the value is not valid.
> >>
> >> So if hardware behavior is fully defined, why don't you pre-adjust what is
> >> to be written to the value hardware would alter it to?
> >>
> > In previous version of MBA patch set, I pre-adjust the value in 'mba_check_thrtl'.
> > But Roger did not like that. So, the pre-adjust codes are removed.
>
> Could you point me at or repeat the reason(s) of his dislike?
>
Roger has replied.
> >> >This check is not only for MBA but also for CAT features that the HW
> >> >cannot alter CAT value.
> >>
> >> I don't understand this part.
> >>
> > I mean the check here are for all features so we cannot remove it.
>
> I _still_ don't understand: If the check can't be removed (even
> without MBA in mind), then the implication would be that the
> code prior to this series is buggy. In which case I'd expect you to
> submit a standalone bug fix, rather than mixing the fix into here.
>
Ok, I will send out a stand alone patch to fix this.
> >> > Although this check is not a critical check,
> >> >it can prevent some non-necessary MSR write.
> >>
> >> That's my point - while previously all unnecessary writes were avoided,
> >> you now avoid only some.
> >>
> > Without the pre-adjust codes in 'mba_check_thrtl', if user inputs value, e.g.
> > 11/22/33/..., this check cannot prevent the write action. So, only some can
> > be avoided in current codes.
>
> Right. If it's worthwhile avoiding the writes, all of them should be
> avoided when the resulting value isn't going to change. Otherwise
> the write avoidance logic can/should be dropped altogether.
>
Per discussion in other mails, I think I will restore codes in 'mba_check_thrtl'.
> Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
end of thread, other threads:[~2017-10-05 10:33 UTC | newest]
Thread overview: 64+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-09-23 9:48 [PATCH v4 00/15] Enable Memory Bandwidth Allocation in Xen Yi Sun
2017-09-23 9:48 ` [PATCH v4 01/15] docs: create Memory Bandwidth Allocation (MBA) feature document Yi Sun
2017-09-25 15:13 ` Roger Pau Monné
2017-09-23 9:48 ` [PATCH v4 02/15] Rename PSR sysctl/domctl interfaces and xsm policy to make them be general Yi Sun
2017-09-25 16:15 ` Roger Pau Monné
2017-09-26 5:15 ` Yi Sun
2017-09-26 14:19 ` Wei Liu
2017-09-28 2:12 ` Yi Sun
2017-09-28 8:34 ` Wei Liu
2017-09-28 10:21 ` Jan Beulich
2017-09-29 1:34 ` Yi Sun
2017-10-04 5:47 ` Jan Beulich
2017-09-23 9:48 ` [PATCH v4 03/15] x86: rename 'cbm_type' to 'psr_type' to make it general Yi Sun
2017-09-28 10:25 ` Jan Beulich
2017-09-29 1:35 ` Yi Sun
2017-09-23 9:48 ` [PATCH v4 04/15] x86: implement data structure and CPU init flow for MBA Yi Sun
2017-09-26 8:38 ` Roger Pau Monné
2017-09-28 10:27 ` Jan Beulich
2017-09-28 11:00 ` Jan Beulich
2017-09-29 1:53 ` Yi Sun
2017-10-04 5:52 ` Jan Beulich
2017-10-05 4:42 ` Yi Sun
2017-10-05 8:49 ` Jan Beulich
2017-10-05 10:08 ` Yi Sun
2017-09-23 9:48 ` [PATCH v4 05/15] x86: implement get hw info " Yi Sun
2017-09-26 8:46 ` Roger Pau Monné
2017-09-26 11:49 ` Jan Beulich
2017-09-23 9:48 ` [PATCH v4 06/15] x86: implement get value interface " Yi Sun
2017-09-28 11:12 ` Jan Beulich
2017-09-23 9:48 ` [PATCH v4 07/15] x86: implement set value flow " Yi Sun
2017-09-26 9:39 ` Roger Pau Monné
2017-09-28 2:39 ` Yi Sun
2017-09-28 11:15 ` Jan Beulich
2017-09-28 11:36 ` Jan Beulich
2017-09-29 2:56 ` Yi Sun
2017-10-04 5:59 ` Jan Beulich
2017-10-05 4:48 ` Yi Sun
2017-10-05 8:39 ` Roger Pau Monné
2017-10-05 9:39 ` Jan Beulich
2017-10-05 10:10 ` Yi Sun
2017-10-05 8:55 ` Jan Beulich
2017-10-05 10:32 ` Yi Sun
2017-09-23 9:48 ` [PATCH v4 08/15] tools: create general interfaces to support psr allocation features Yi Sun
2017-09-26 10:11 ` Roger Pau Monné
2017-09-23 9:48 ` [PATCH v4 09/15] tools: implement the new libxc get hw info interface Yi Sun
2017-09-26 10:22 ` Roger Pau Monné
2017-09-23 9:48 ` [PATCH v4 10/15] tools: implement the new libxl " Yi Sun
2017-09-26 10:54 ` Roger Pau Monné
2017-09-23 9:48 ` [PATCH v4 11/15] tools: implement the new xl " Yi Sun
2017-09-26 11:19 ` Roger Pau Monné
2017-09-23 9:48 ` [PATCH v4 12/15] tools: rename 'xc_psr_cat_type' to 'xc_psr_type' Yi Sun
2017-09-26 11:21 ` Roger Pau Monné
2017-09-23 9:48 ` [PATCH v4 13/15] tools: implement new generic get value interface and MBA get value command Yi Sun
2017-09-26 11:34 ` Roger Pau Monné
2017-09-23 9:48 ` [PATCH v4 14/15] tools: implement new generic set value interface and MBA set " Yi Sun
2017-09-26 11:39 ` Roger Pau Monné
2017-09-28 2:46 ` Yi Sun
2017-09-23 9:48 ` [PATCH v4 15/15] docs: add MBA description in docs Yi Sun
2017-09-26 11:45 ` Roger Pau Monné
2017-09-26 11:48 ` [PATCH v4 00/15] Enable Memory Bandwidth Allocation in Xen Roger Pau Monné
2017-09-28 2:20 ` Yi Sun
2017-09-28 15:57 ` Wei Liu
2017-09-29 2:58 ` Yi Sun
2017-09-29 8:02 ` Wei Liu
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.