Xen-Devel Archive on lore.kernel.org
 help / color / Atom feed
From: "Roger Pau Monné" <roger.pau@citrix.com>
To: Chao Gao <chao.gao@intel.com>
Cc: xen-devel@lists.xenproject.org, Jan Beulich <jbeulich@suse.com>,
	Ashok Raj <ashok.raj@intel.com>, Wei Liu <wl@xen.org>,
	Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [Xen-devel] [PATCH v9 04/15] microcode: introduce a global cache of ucode patch
Date: Thu, 22 Aug 2019 13:11:04 +0200
Message-ID: <20190822111104.f34725kacyiu63oy@Air-de-Roger> (raw)
In-Reply-To: <1566177928-19114-5-git-send-email-chao.gao@intel.com>

On Mon, Aug 19, 2019 at 09:25:17AM +0800, Chao Gao wrote:
> to replace the current per-cpu cache 'uci->mc'.
> 
> With the assumption that all CPUs in the system have the same signature
> (family, model, stepping and 'pf'), one microcode update matches with
> one cpu should match with others. Having multiple microcode revisions
> on different cpus would cause system unstable and should be avoided.
> Hence, caching only one microcode update is good enough for all cases.
> 
> Introduce a global variable, microcode_cache, to store the newest
> matching microcode update. Whenever we get a new valid microcode update,
> its revision id is compared against that of the microcode update to
> determine whether the "microcode_cache" needs to be replaced. And
> this global cache is loaded to cpu in apply_microcode().
> 
> All operations on the cache is protected by 'microcode_mutex'.
> 
> Note that I deliberately avoid touching the old per-cpu cache ('uci->mc')
> as I am going to remove it completely in the following patches. We copy
> everything to create the new cache blob to avoid reusing some buffers
> previously allocated for the old per-cpu cache. It is not so efficient,
> but it is already corrected by a patch later in this series.
> 
> Signed-off-by: Chao Gao <chao.gao@intel.com>

Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>

I have some nits below, but I don't think they affect functionality in
anyway, hence my RB. It would be nice to get those fixed as follow-ups
if others think the current version is suitable for committing.

> ---
> Changes in v9:
>  - on Intel side, ->compare_patch just checks the patch revision number.
>  - explain why all buffers are copied in alloc_microcode_patch() in
>  patch description.
> 
> Changes in v8:
>  - Free generic wrapper struct in general code
>  - Try to update cache as long as a patch covers current cpu. Previsouly,
>  cache is updated only if the patch is newer than current update revision in
>  the CPU. The small difference can work around a broken bios which only
>  applies microcode update to BSP and software has to apply the same
>  update to other CPUs.
> 
> Changes in v7:
>  - reworked to cache only one microcode patch rather than a list of
>  microcode patches.
> ---
>  xen/arch/x86/microcode.c        | 39 ++++++++++++++++++
>  xen/arch/x86/microcode_amd.c    | 90 +++++++++++++++++++++++++++++++++++++----
>  xen/arch/x86/microcode_intel.c  | 73 ++++++++++++++++++++++++++-------
>  xen/include/asm-x86/microcode.h | 17 ++++++++
>  4 files changed, 197 insertions(+), 22 deletions(-)
> 
> diff --git a/xen/arch/x86/microcode.c b/xen/arch/x86/microcode.c
> index 421d57e..0ecd2fd 100644
> --- a/xen/arch/x86/microcode.c
> +++ b/xen/arch/x86/microcode.c
> @@ -61,6 +61,9 @@ static struct ucode_mod_blob __initdata ucode_blob;
>   */
>  static bool_t __initdata ucode_scan;
>  
> +/* Protected by microcode_mutex */
> +static struct microcode_patch *microcode_cache;
> +
>  void __init microcode_set_module(unsigned int idx)
>  {
>      ucode_mod_idx = idx;
> @@ -262,6 +265,42 @@ int microcode_resume_cpu(unsigned int cpu)
>      return err;
>  }
>  
> +void microcode_free_patch(struct microcode_patch *microcode_patch)
> +{
> +    microcode_ops->free_patch(microcode_patch->mc);
> +    xfree(microcode_patch);
> +}
> +
> +const struct microcode_patch *microcode_get_cache(void)
> +{
> +    ASSERT(spin_is_locked(&microcode_mutex));
> +
> +    return microcode_cache;
> +}
> +
> +/* Return true if cache gets updated. Otherwise, return false */
> +bool microcode_update_cache(struct microcode_patch *patch)
> +{
> +

Nit: extra newline above.

> +    ASSERT(spin_is_locked(&microcode_mutex));
> +
> +    if ( !microcode_cache )
> +        microcode_cache = patch;
> +    else if ( microcode_ops->compare_patch(patch,
> +                                           microcode_cache) == NEW_UCODE )
> +    {
> +        microcode_free_patch(microcode_cache);
> +        microcode_cache = patch;
> +    }
> +    else
> +    {
> +        microcode_free_patch(patch);
> +        return false;
> +    }
> +
> +    return true;
> +}
> +
>  static int microcode_update_cpu(const void *buf, size_t size)
>  {
>      int err;
> diff --git a/xen/arch/x86/microcode_amd.c b/xen/arch/x86/microcode_amd.c
> index 3db3555..30129ca 100644
> --- a/xen/arch/x86/microcode_amd.c
> +++ b/xen/arch/x86/microcode_amd.c
> @@ -190,24 +190,83 @@ static enum microcode_match_result microcode_fits(
>      return NEW_UCODE;
>  }
>  
> +static bool match_cpu(const struct microcode_patch *patch)
> +{
> +    if ( !patch )
> +        return false;
> +    return microcode_fits(patch->mc_amd, smp_processor_id()) == NEW_UCODE;
> +}
> +
> +static struct microcode_patch *alloc_microcode_patch(
> +    const struct microcode_amd *mc_amd)
> +{
> +    struct microcode_patch *microcode_patch = xmalloc(struct microcode_patch);
> +    struct microcode_amd *cache = xmalloc(struct microcode_amd);
> +    void *mpb = xmalloc_bytes(mc_amd->mpb_size);
> +    struct equiv_cpu_entry *equiv_cpu_table =
> +                                xmalloc_bytes(mc_amd->equiv_cpu_table_size);
> +
> +    if ( !microcode_patch || !cache || !mpb || !equiv_cpu_table )
> +    {
> +        xfree(microcode_patch);
> +        xfree(cache);
> +        xfree(mpb);
> +        xfree(equiv_cpu_table);
> +        return ERR_PTR(-ENOMEM);
> +    }
> +
> +    memcpy(mpb, mc_amd->mpb, mc_amd->mpb_size);
> +    cache->mpb = mpb;
> +    cache->mpb_size = mc_amd->mpb_size;
> +    memcpy(equiv_cpu_table, mc_amd->equiv_cpu_table,
> +           mc_amd->equiv_cpu_table_size);
> +    cache->equiv_cpu_table = equiv_cpu_table;
> +    cache->equiv_cpu_table_size = mc_amd->equiv_cpu_table_size;
> +    microcode_patch->mc_amd = cache;
> +
> +    return microcode_patch;
> +}
> +
> +static void free_patch(void *mc)
> +{
> +    struct microcode_amd *mc_amd = mc;
> +
> +    xfree(mc_amd->equiv_cpu_table);
> +    xfree(mc_amd->mpb);
> +    xfree(mc_amd);
> +}

It's asymmetric that alloc_microcode_patch allocates microcode_patch,
but free_patch doesn't free it. Not a big deal, but it would be good
to make this symmetric IMO.

> +
> +static enum microcode_match_result compare_patch(
> +    const struct microcode_patch *new, const struct microcode_patch *old)
> +{
> +    const struct microcode_amd *new_mc = new->mc_amd;
> +    const struct microcode_header_amd *new_header = new_mc->mpb;
> +    const struct microcode_amd *old_mc = old->mc_amd;
> +    const struct microcode_header_amd *old_header = old_mc->mpb;

The local variables new_mc/old_mc are used just one, and hence are
not really helpful IMO, you could just do:

const struct microcode_header_amd *new_header = new->mc_amd->mpb;
const struct microcode_header_amd *old_header = old->mc_amd->mpb;

Again, just a nit.

> +
> +    if ( new_header->processor_rev_id == old_header->processor_rev_id )
> +        return (new_header->patch_id > old_header->patch_id) ?
> +                NEW_UCODE : OLD_UCODE;
> +
> +    return MIS_UCODE;
> +}
> +
>  static int apply_microcode(unsigned int cpu)
>  {
>      unsigned long flags;
>      struct ucode_cpu_info *uci = &per_cpu(ucode_cpu_info, cpu);
>      uint32_t rev;
> -    struct microcode_amd *mc_amd = uci->mc.mc_amd;
> -    struct microcode_header_amd *hdr;
>      int hw_err;
> +    const struct microcode_header_amd *hdr;
> +    const struct microcode_patch *patch = microcode_get_cache();
>  
>      /* We should bind the task to the CPU */
>      BUG_ON(raw_smp_processor_id() != cpu);
>  
> -    if ( mc_amd == NULL )
> +    if ( !match_cpu(patch) )
>          return -EINVAL;

Another nit, but !patch should get ENOENT rather than EINVAL IMO.

>  
> -    hdr = mc_amd->mpb;
> -    if ( hdr == NULL )
> -        return -EINVAL;
> +    hdr = patch->mc_amd->mpb;
>  
>      spin_lock_irqsave(&microcode_update_lock, flags);
>  
> @@ -496,7 +555,21 @@ static int cpu_request_microcode(unsigned int cpu, const void *buf,
>      while ( (error = get_ucode_from_buffer_amd(mc_amd, buf, bufsize,
>                                                 &offset)) == 0 )
>      {
> -        if ( microcode_fits(mc_amd, cpu) == NEW_UCODE )
> +        struct microcode_patch *new_patch = alloc_microcode_patch(mc_amd);
> +
> +        if ( IS_ERR(new_patch) )
> +        {
> +            error = PTR_ERR(new_patch);
> +            break;
> +        }
> +
> +        /* Update cache if this patch covers current CPU */
> +        if ( microcode_fits(new_patch->mc_amd, cpu) != MIS_UCODE )
> +            microcode_update_cache(new_patch);
> +        else
> +            microcode_free_patch(new_patch);
> +
> +        if ( match_cpu(microcode_get_cache()) )
>          {
>              error = apply_microcode(cpu);
>              if ( error )
> @@ -640,6 +713,9 @@ static const struct microcode_ops microcode_amd_ops = {
>      .collect_cpu_info                 = collect_cpu_info,
>      .apply_microcode                  = apply_microcode,
>      .start_update                     = start_update,
> +    .free_patch                       = free_patch,
> +    .compare_patch                    = compare_patch,
> +    .match_cpu                        = match_cpu,
>  };
>  
>  int __init microcode_init_amd(void)
> diff --git a/xen/arch/x86/microcode_intel.c b/xen/arch/x86/microcode_intel.c
> index c185b5c..14485dc 100644
> --- a/xen/arch/x86/microcode_intel.c
> +++ b/xen/arch/x86/microcode_intel.c
> @@ -259,6 +259,31 @@ static int microcode_sanity_check(void *mc)
>      return 0;
>  }
>  
> +static bool match_cpu(const struct microcode_patch *patch)
> +{
> +    if ( !patch )
> +        return false;
> +
> +    return microcode_update_match(&patch->mc_intel->hdr,
> +                                  smp_processor_id()) == NEW_UCODE;
> +}
> +
> +static void free_patch(void *mc)
> +{
> +    xfree(mc);
> +}
> +
> +/*
> + * Both patches to compare are supposed to be applicable to local CPU.
> + * Just compare the revision number.
> + */
> +static enum microcode_match_result compare_patch(
> +    const struct microcode_patch *new, const struct microcode_patch *old)
> +{
> +    return (new->mc_intel->hdr.rev > old->mc_intel->hdr.rev) ?  NEW_UCODE :
> +                                                                OLD_UCODE;

Nit, the usual format in Xen would be:

return (new->mc_intel->hdr.rev > old->mc_intel->hdr.rev) ? NEW_UCODE
                                                         : OLD_UCODE;

> +}
> +
>  /*
>   * return 0 - no update found
>   * return 1 - found update
> @@ -269,10 +294,26 @@ static int get_matching_microcode(const void *mc, unsigned int cpu)
>      struct ucode_cpu_info *uci = &per_cpu(ucode_cpu_info, cpu);
>      const struct microcode_header_intel *mc_header = mc;
>      unsigned long total_size = get_totalsize(mc_header);
> -    void *new_mc;
> +    void *new_mc = xmalloc_bytes(total_size);
> +    struct microcode_patch *new_patch = xmalloc(struct microcode_patch);
>  
> -    if ( microcode_update_match(mc, cpu) != NEW_UCODE )
> +    if ( !new_patch || !new_mc )
> +    {
> +        xfree(new_patch);
> +        xfree(new_mc);
> +        return -ENOMEM;
> +    }
> +    memcpy(new_mc, mc, total_size);
> +    new_patch->mc_intel = new_mc;
> +
> +    /* Make sure that this patch covers current CPU */
> +    if ( microcode_update_match(mc, cpu) == MIS_UCODE )
> +    {
> +        microcode_free_patch(new_patch);
>          return 0;
> +    }

Nit: won't it be easier to do this check first and then allocate if
needed?

AFAICT new_mc or new_patch are not required by the
microcode_update_match call, and hence could be allocated after such
call. Maybe this ugliness will go away in following patches?

Thanks, Roger.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

  reply index

Thread overview: 57+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-08-19  1:25 [Xen-devel] [PATCH v9 00/15] improve late microcode loading Chao Gao
2019-08-19  1:25 ` [Xen-devel] [PATCH v9 01/15] microcode/intel: extend microcode_update_match() Chao Gao
2019-08-28 15:12   ` Jan Beulich
2019-08-29  7:15     ` Chao Gao
2019-08-29  7:14       ` Jan Beulich
2019-08-19  1:25 ` [Xen-devel] [PATCH v9 02/15] microcode/amd: fix memory leak Chao Gao
2019-08-19  1:25 ` [Xen-devel] [PATCH v9 03/15] microcode/amd: distinguish old and mismatched ucode in microcode_fits() Chao Gao
2019-08-19  1:25 ` [Xen-devel] [PATCH v9 04/15] microcode: introduce a global cache of ucode patch Chao Gao
2019-08-22 11:11   ` Roger Pau Monné [this message]
2019-08-28 15:21   ` Jan Beulich
2019-08-29 10:18   ` Jan Beulich
2019-08-19  1:25 ` [Xen-devel] [PATCH v9 05/15] microcode: clean up microcode_resume_cpu Chao Gao
2019-08-19  1:25 ` [Xen-devel] [PATCH v9 06/15] microcode: remove struct ucode_cpu_info Chao Gao
2019-08-19  1:25 ` [Xen-devel] [PATCH v9 07/15] microcode: remove pointless 'cpu' parameter Chao Gao
2019-08-19  1:25 ` [Xen-devel] [PATCH v9 08/15] microcode/amd: call svm_host_osvw_init() in common code Chao Gao
2019-08-22 13:08   ` Roger Pau Monné
2019-08-28 15:26   ` Jan Beulich
2019-08-19  1:25 ` [Xen-devel] [PATCH v9 09/15] microcode: pass a patch pointer to apply_microcode() Chao Gao
2019-08-19  1:25 ` [Xen-devel] [PATCH v9 10/15] microcode: split out apply_microcode() from cpu_request_microcode() Chao Gao
2019-08-22 13:59   ` Roger Pau Monné
2019-08-29 10:06     ` Jan Beulich
2019-08-30  3:22       ` Chao Gao
2019-08-30  7:25         ` Jan Beulich
2019-08-29 10:19   ` Jan Beulich
2019-08-19  1:25 ` [Xen-devel] [PATCH v9 11/15] microcode: unify loading update during CPU resuming and AP wakeup Chao Gao
2019-08-22 14:10   ` Roger Pau Monné
2019-08-22 16:44     ` Chao Gao
2019-08-23  9:09       ` Roger Pau Monné
2019-08-29  7:37         ` Chao Gao
2019-08-29  8:16           ` Roger Pau Monné
2019-08-29 10:26           ` Jan Beulich
2019-08-29 10:29   ` Jan Beulich
2019-08-19  1:25 ` [Xen-devel] [PATCH v9 12/15] microcode: reduce memory allocation and copy when creating a patch Chao Gao
2019-08-23  8:11   ` Roger Pau Monné
2019-08-26  7:03     ` Chao Gao
2019-08-26  8:11       ` Roger Pau Monné
2019-08-29 10:47   ` Jan Beulich
2019-08-19  1:25 ` [Xen-devel] [PATCH v9 13/15] x86/microcode: Synchronize late microcode loading Chao Gao
2019-08-19 10:27   ` Sergey Dyasli
2019-08-19 14:49     ` Chao Gao
2019-08-29 12:06   ` Jan Beulich
2019-08-30  3:30     ` Chao Gao
2019-08-19  1:25 ` [Xen-devel] [PATCH v9 14/15] microcode: remove microcode_update_lock Chao Gao
2019-08-19  1:25 ` [Xen-devel] [PATCH v9 15/15] microcode: block #NMI handling when loading an ucode Chao Gao
2019-08-23  8:46   ` Sergey Dyasli
2019-08-26  8:07     ` Chao Gao
2019-08-27  4:52       ` Chao Gao
2019-08-28  8:52         ` Sergey Dyasli
2019-08-29 12:11         ` Jan Beulich
2019-08-30  6:35           ` Chao Gao
2019-09-09  5:52             ` Chao Gao
2019-09-09  6:16               ` Jan Beulich
2019-08-29 12:22   ` Jan Beulich
2019-08-30  6:33     ` Chao Gao
2019-08-30  7:30       ` Jan Beulich
2019-08-22  7:51 ` [Xen-devel] [PATCH v9 00/15] improve late microcode loading Sergey Dyasli
2019-08-22 15:39   ` Chao Gao

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20190822111104.f34725kacyiu63oy@Air-de-Roger \
    --to=roger.pau@citrix.com \
    --cc=andrew.cooper3@citrix.com \
    --cc=ashok.raj@intel.com \
    --cc=chao.gao@intel.com \
    --cc=jbeulich@suse.com \
    --cc=wl@xen.org \
    --cc=xen-devel@lists.xenproject.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Xen-Devel Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/xen-devel/0 xen-devel/git/0.git
	git clone --mirror https://lore.kernel.org/xen-devel/1 xen-devel/git/1.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 xen-devel xen-devel/ https://lore.kernel.org/xen-devel \
		xen-devel@lists.xenproject.org xen-devel@lists.xen.org
	public-inbox-index xen-devel

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.xenproject.lists.xen-devel


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git