[PATCH v10 00/25] Enable L2 Cache Allocation Technology & Refactor psr.c

[Yi Sun <yi.y.sun@linux.intel.com>]

Hi all,

We plan to bring a new PSR (Platform Shared Resource) feature called
Intel L2 Cache Allocation Technology (L2 CAT) to Xen. It has been enabled
in Linux Kernel.

Besides the L2 CAT implementaion, we refactor the psr.c to make it more
flexible and easily to extend to add new features. We abstract the general
operations of all features and encapsulate them into a structure. Then,
the development of new feature is simple to mainly implement these callback
functions.

The patch set can be found at:
https://github.com/yisun-git/xen.git l2_cat_v10

---
Acked and Reviewed list before V10:

a - Acked-by
r - Reviewed-by

  r  patch 1  - docs: create Cache Allocation Technology (CAT) and Code and
                Data Prioritization (CDP) feature document
  ar patch 2  - x86: refactor psr: remove L3 CAT/CDP codes.
  a  patch 4  - x86: move cpuid_count_leaf from cpuid.c to processor.h.
  a  patch 25 - docs: add L2 CAT description in docs.

---
V10 change list:

Patch 3:
    - remove initialization for 'PSR_SOCKET_L3_CAT'.
      (suggested by Jan Beulich)
    - rename 'feat_ops' to 'feat_props'.
      (suggested by Jan Beulich)
    - move 'cbm_len' to 'feat_props' because it is feature independent so far.
      (suggested by Jan Beulich)
    - move 'cos_max' to 'feat_props' because it is feature independent.
      (suggested by Jan Beulich)
    - move 'cos_num' to 'feat_props' because it is feature independent.
      (suggested by Jan Beulich)
    - remove union 'info' and struct 'psr_cat_hw_info'.
    - remove 'get_cos_max' from 'feat_props'.
      (suggested by Jan Beulich)
    - remove 'feat_mask' from 'psr_socket_info' because we can use 'features[]'
      to check if any feature is initialized.
      (suggested by Jan Beulich)
    - move 'ref_lock' above 'cos_ref'.
      (suggested by Jan Beulich)
    - adjust comments and commit message according to above changes.
Patch 4:
    - Acked by Jan.
Patch 5:
    - remove 'asm/x86_emulate.h' inclusion as it has been indirectly included.
      (suggested by Jan Beulich)
    - remove 'CAT_COS_NUM' as it is only used once.
      (suggested by Jan Beulich)
    - remove 'feat_mask'.
      (suggested by Jan Beulich)
    - changes about 'feat_props'.
      (suggested by Jan Beulich)
    - remove 'get_cos_max' hook declaration.
      (suggested by Jan Beulich)
    - modify 'cat_default_val' implementation.
      (suggested by Jan Beulich)
    - modify 'psr_alloc_feat_enabled' implementation to make it simple.
      (suggested by Jan Beulich)
    - rename 'free_feature' to 'free_socket_resources' because it is executed
      when socket is offline. It needs free resources related to the socket.
      (suggested by Jan Beulich)
    - define 'feat_init_done' to iterate feature array to check if any feature
      has been initialized.
      (suggested by Jan Beulich)
    - input 'struct cpuid_leaf' pointer into 'cat_init_feature' to avoid memory
      copy.
      (suggested by Jan Beulich)
    - modify 'cat_init_feature' to use switch and things related to above
      changes.
      (suggested by Jan Beulich)
    - add an indentation for label.
      (suggested by Jan Beulich)
Patch 6:
    - remove 'cat_get_cos_max' as 'cos_max' is a feature property now which
      can be directly used.
      (suggested by Jan Beulich)
    - replace 'info->feat_mask' check to ''feat_init_done'.
      (suggested by Jan Beulich)
Patch 7:
    - remove 'PSR_SOCKET_UNKNOWN' and use 'ASSERT_UNREACHABLE()' to handle
      this case.
      (suggested by Jan Beulich)
    - check 'feat_type'.
      (suggested by Jan Beulich)
    - adjust macros names and values to make them more appropriate.
      (suggested by Jan Beulich)
    - use 'feat_init_done'.
      (suggested by Jan Beulich)
    - changes about 'cbm_len'.
      (suggested by Jan Beulich)
Patch 8:
    - use an intermediate variable to get value and avoid cast in domctl.
      (suggested by Jan Beulich)
    - remove 'type' in 'get_val' parameters and will add it back when
      implementing CDP.
      (suggested by Jan Beulich)
    - add 'err' in 'psr_get_feat' parameters to get error number back.
      (suggested by Jan Beulich)
    - remove unnecessary variable in 'psr_get_feat'.
      (suggested by Jan Beulich)
    - use 'ASSET' to check input parameter in 'psr_get_val'.
      (suggested by Jan Beulich)
    - changes about 'feat_props'.
      (suggested by Jan Beulich)
Patch 9:
    - restore domain cos id to 0 when socket is offline.
      (suggested by Jan Beulich)
    - check 'psr_cat_op.data' to make sure only lower 32 bits are valid.
      (suggested by Jan Beulich)
    - remove unnecessary fixed width type of parameters and variables.
      (suggested by Jan Beulich)
    - rename 'insert_new_val_to_array' to 'insert_val_to_array'.
      (suggested by Jan Beulich)
    - input 'ref_lock' pointer into functions to check if it has been locked.
      (suggested by Jan Beulich)
    - add comment to declare the set process is protected by 'domctl_lock'.
      (suggested by Jan Beulich)
    - check 'feat_type'.
      (suggested by Jan Beulich)
    - remove 'feat_mask'.
      (suggested by Jan Beulich)
    - remove unnecessary criteria of ASSERT.
      (suggested by Jan Beulich)
    - adjust flow of 'psr_set_val' to avoid 'goto' for successful cases.
      (suggested by Jan Beulich)
    - use ASSERT to check 'socket_info' in 'psr_free_cos'.
      (suggested by Jan Beulich)
    - remove unnecessary comment in 'psr_free_cos'.
      (suggested by Jan Beulich)
Patch 10:
    - remove 'get_old_val' to directly call 'get_val' to get needed val.
      (suggested by Jan Beulich)
    - move 'psr_check_cbm' into 'insert_val_to_array'.
      (suggested by Jan Beulich)
    - change type of 'cbm' in 'psr_check_cbm' to 'unsigned long'.
      (suggested by Jan Beulich)
    - remove 'set_new_val' as it can be handled in generic process.
    - changes related to 'feat_props'.
      (suggested by Jan Beulich)
    - adjust flow in 'gather_val_array' to avoid array cross.
      (suggested by Jan Beulich)
    - adjust flow in 'insert_val_to_array' to avoid array cross.
      (suggested by Jan Beulich)
Patch 11:
    - remove 'compare_val' hook and its CAT implementation. Make its
      functionality be generic in 'find_cos' flow.
      (suggested by Jan Beulich)
    - changes related to 'props'.
      (suggested by Jan Beulich)
    - rename 'val_array' to 'val_ptr'.
      (suggested by Jan Beulich)
    - rename 'find' to 'found'.
      (suggested by Jan Beulich)
    - move some variables declaration and initialization into loop.
      (suggested by Jan Beulich)
    - adjust codes positions.
      (suggested by Jan Beulich)
Patch 12:
    - remove 'fits_cos_max' hook and CAT implementation. Move the process into
      generic flow.
      (suggested by Jan Beulich)
    - changes about 'props'.
      (suggested by Jan Beulich)
    - adjust codes positions.
      (suggested by Jan Beulich)
Patch 13:
    - remove 'type' from 'write_msr' parameter list. Will add it back when
      implementing CDP.
      (suggested by Jan Beulich)
    - remove unnecessary casts.
      (suggested by Jan Beulich)
    - changes about 'props'.
      (suggested by Jan Beulich)
Patch 14:
    - fix comment.
      (suggested by Jan Beulich)
    - use swith in 'cat_init_feature' to handle different feature types.
      (suggested by Jan Beulich)
    - changes about 'props'.
      (suggested by Jan Beulich)
    - restore MSRs to default value when cpu online.
      (suggested by Jan Beulich)
    - remove feat_mask.
      (suggested by Jan Beulich)
Patch 15:
    - update renamed macros used by psr_get_info.
      (suggested by Jan Beulich)
    - change 'psr_get_info' flow to cover CDP case to make codes in sysctl
      more simple.
      (suggested by Jan Beulich)
    - remove sysctl redundant codes after applying above changes.
      (suggested by Jan Beulich)
Patch 16:
    - add 'enum cbm_type type' into 'get_val' parameters to handle CDP case.
      (suggested by Jan Beulich)
Patch 17:
    - remove 'l3_cdp_get_old_val' and use 'l3_cdp_get_val' to replace it.
      (suggested by Jan Beulich)
    - remvoe 'l3_cdp_set_new_val'.
    - modify 'insert_val_to_array' flow to handle multiple COSs case.
      (suggested by Jan Beulich)
    - remove 'l3_cdp_compare_val' and implement a generic function
      'comapre_val'.
      (suggested by Jan Beulich)
    - remove 'l3_cdp_fits_cos_max'.
      (suggested by Jan Beulich)
    - introduce macro 'PSR_MAX_COS_NUM'.
    - introduce a new member in 'feat_props', 'type[PSR_MAX_COS_NUM]' to record
      all 'cbm_type' the feature has.
      (suggested by Jan Beulich)
    - modify 'gather_val_array' flow to handle multiple COSs case.
      (suggested by Jan Beulich)
    - modify 'find_cos' flow and implement 'compare_val' to handle multiple
      COSs case.
      (suggested by Jan Beulich)
    - modify 'fits_cos_max' flow to handle multiple COSs case.
      (suggested by Jan Beulich)
    - changes about 'props'.
      (suggested by Jan Beulich)
    - remove cast in 'l3_cdp_write_msr'.
      (suggested by Jan Beulich)
    - implement 'compare_val' function to compare if feature values are what
      we expect in finding flow.
    - implement 'restore_default_val' function to restore feature's COS values
      to default if the feature has multiple COSs. It is called when the COS
      ID is reduced to 0.
Patch 18:
    - implement L2 CAT case in 'cat_init_feature'.
      (suggested by Jan Beulich)
    - changes about 'props'.
      (suggested by Jan Beulich)
    - introduce 'PSR_CBM_TYPE_L2'.
Patch 19:
    - modify macro name according to previous patch change.
      (suggested by Jan Beulich)
Patch 20:
    - remove cast in domctl.
      (suggested by Jan Beulich)
Patch 21:
    - check input data and remove cast in domctl.
      (suggested by Jan Beulich)
    - remove some hooks assignment due to previous patches changes.
      (suggested by Jan Beulich)
    - remove cast in 'l2_cat_write_msr'.
      (suggested by Jan Beulich)
Patch 22:
    - change macros names according to previous changes.
      (suggested by Jan Beulich)
Patch 24:
    - fix comments.
      (suggested by Wei Liu)

Yi Sun (25):
  docs: create Cache Allocation Technology (CAT) and Code and Data
    Prioritization (CDP) feature document
  x86: refactor psr: remove L3 CAT/CDP codes.
  x86: refactor psr: implement main data structures.
  x86: move cpuid_count_leaf from cpuid.c to processor.h.
  x86: refactor psr: L3 CAT: implement CPU init and free flow.
  x86: refactor psr: L3 CAT: implement Domain init/free and schedule
    flows.
  x86: refactor psr: L3 CAT: implement get hw info flow.
  x86: refactor psr: L3 CAT: implement get value flow.
  x86: refactor psr: L3 CAT: set value: implement framework.
  x86: refactor psr: L3 CAT: set value: assemble features value array.
  x86: refactor psr: L3 CAT: set value: implement cos finding flow.
  x86: refactor psr: L3 CAT: set value: implement cos id picking flow.
  x86: refactor psr: L3 CAT: set value: implement write msr flow.
  x86: refactor psr: CDP: implement CPU init and free flow.
  x86: refactor psr: CDP: implement get hw info flow.
  x86: refactor psr: CDP: implement get value flow.
  x86: refactor psr: CDP: implement set value callback functions.
  x86: L2 CAT: implement CPU init and free flow.
  x86: L2 CAT: implement get hw info flow.
  x86: L2 CAT: implement get value flow.
  x86: L2 CAT: implement set value flow.
  tools: L2 CAT: support get HW info for L2 CAT.
  tools: L2 CAT: support show cbm for L2 CAT.
  tools: L2 CAT: support set cbm for L2 CAT.
  docs: add L2 CAT description in docs.

 docs/features/intel_psr_cat_cdp.pandoc |  469 +++++++++++
 docs/man/xl.pod.1.in                   |   25 +-
 docs/misc/xl-psr.markdown              |   18 +-
 tools/libxc/include/xenctrl.h          |    7 +-
 tools/libxc/xc_psr.c                   |   45 +-
 tools/libxl/libxl.h                    |    9 +
 tools/libxl/libxl_psr.c                |   68 +-
 tools/libxl/libxl_types.idl            |    1 +
 tools/xl/xl_cmdtable.c                 |    6 +-
 tools/xl/xl_psr.c                      |  168 ++--
 xen/arch/x86/cpuid.c                   |    6 -
 xen/arch/x86/domctl.c                  |   81 +-
 xen/arch/x86/psr.c                     | 1450 +++++++++++++++++++++++++-------
 xen/arch/x86/sysctl.c                  |   40 +-
 xen/include/asm-x86/msr-index.h        |    1 +
 xen/include/asm-x86/processor.h        |    7 +
 xen/include/asm-x86/psr.h              |   26 +-
 xen/include/public/domctl.h            |    2 +
 xen/include/public/sysctl.h            |    3 +-
 19 files changed, 1989 insertions(+), 443 deletions(-)
 create mode 100644 docs/features/intel_psr_cat_cdp.pandoc

-- 
1.9.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel