question on microcode loading

question on microcode loading

[Steffen Nurpmeso]

Hello!

First a thank you for kexec, i use it for a homegrown boot
solution, where stage1 EFI_STUB kernel is used with busybox and
cryptsetup to unlock an encrypted device, and then boots into
stage2 via kexec -- a wonderful and easy solution that makes me
happy (for years)!

A bit lengthy now.
There is only one question regarding CPU microcode loading: it
seems the microcode is not "updated early" with kexec upon "boot".
This is a homegrown kernel, with built-in firmware

  CONFIG_EXTRA_FIRMWARE="intel-ucode/06-8e-0a ..."

but also with the firmware (prefix)in(g) the initrd

    {
            # Microcode update must be uncompressed and first
            [ -f ../../boot/early-ucode.cpio ] && ./$BB cat ../../boot/early-ucode.cpio
            # Followed by (possibly compressed) normal initrd
            ./$BB find . | ./$BB cpio -H newc -o | ./$BB gzip -9 -n
    } > ../.initrd

(ie early-ucode.cpio exists) like here:

  #?0|kent:/boot# cpio -t <.kent.initrd.0
  kernel
  kernel/x86
  kernel/x86/microcode
  kernel/x86/microcode/.enuineIntel.align.0123456789abc
  kernel/x86/microcode/GenuineIntel.bin
  11020 blocks
  #?0|kent:/boot# dd if=.kent.initrd.0 skip=5642240 bs=1|gunzip|cpio -t
  .
  init
  linux-init-s1.sh
  sys
  run
  proc
  mnt
  dev
  dev/console
  bin
  bin/sh
  etc
  etc/mdev.sh
  etc/mdev.conf
  linux-init-lib.sh
  linux-init-s2.sh
  cryptsetup.static
  busybox.static
  4491733+0 records in
  4491733+0 records out
  4491733 bytes (4.5 MB, 4.3 MiB) copied, 5.67558 s, 791 kB/s
  18439 blocks

If stage2 then boots it goes like

  Feb  6 17:38:15 (none) kernel: Linux version 6.1.9-ideapad (ports@kent) (gcc (CRUX-x86_64-multilib) 12.2.0, GNU ld (GNU Binutils) 2.39) #1 SMP PREEMPT_DYNAMIC Sat Feb  4 20:40:06 CET 2023
  Feb  6 17:38:15 (none) kernel: Command line: rtw88_pci.disable_aspm=1 rc.hostname=kent
  Feb  6 17:38:15 (none) kernel: KERNEL supported cpus:
  Feb  6 17:38:15 (none) kernel:   Intel GenuineIntel
  ...
  Feb  6 17:38:15 (none) kernel: Freeing SMP alternatives memory: 48K
  Feb  6 17:38:15 (none) kernel: smpboot: CPU0: Intel(R) Core(TM) i5-8250U CPU @ 1.60GHz (family: 0x6, model: 0x8e, stepping: 0xa)
  ..
  Feb  6 17:38:15 (none) kernel: .... node  #0, CPUs:      #1 #2 #3 #4
  Feb  6 17:38:15 (none) kernel: MDS CPU bug present and SMT on, data leak possible. See[..]
  Feb  6 17:38:15 (none) kernel: MMIO Stale Data CPU bug present and SMT on, data leak possible. See[..]
  Feb  6 17:38:15 (none) kernel:  #5 #6 #7
  Feb  6 17:38:15 (none) kernel: smp: Brought up 1 node, 8 CPUs
  Feb  6 17:38:15 (none) kernel: smpboot: Max logical packages: 1
  Feb  6 17:38:15 (none) kernel: smpboot: Total of 8 processors activated (28811.00 BogoMIPS)
  ...
  Feb  6 17:38:15 (none) kernel: Unpacking initramfs...
  ...
  Feb  6 17:38:15 (none) kernel: Freeing initrd memory: 9900K
  ...
  Feb  6 17:38:16 (none) kernel: microcode: sig=0x806ea, pf=0x80, revision=0xf0
  Feb  6 17:38:16 (none) kernel: microcode: Microcode Update Driver: v2.2.

Ie microcode comes late (SMT is later disabled)  It (seems) ok
after suspend/resume (due to
  Feb  7 01:30:12 (none) /root/bin/zzz.sh:
    echo mem > /sys/power/state
  Feb  7 20:11:26 (none) /root/bin/zzz.sh:
    echo off > /sys/devices/system/cpu/smt/control
):

  Feb  7 01:30:12 (none) kernel: PM: suspend entry (deep)
  ...
  Feb  7 20:11:23 (none) kernel: ACPI: PM: Preparing to enter system sleep state S3
  Feb  7 20:11:23 (none) kernel: ACPI: EC: event blocked
  Feb  7 20:11:23 (none) kernel: ACPI: EC: EC stopped
  Feb  7 20:11:23 (none) kernel: ACPI: PM: Saving platform NVS memory
  Feb  7 20:11:23 (none) kernel: Disabling non-boot CPUs ...
  Feb  7 20:11:23 (none) kernel: smpboot: CPU 1 is now offline
  Feb  7 20:11:23 (none) kernel: smpboot: CPU 2 is now offline
  Feb  7 20:11:23 (none) kernel: smpboot: CPU 3 is now offline
  Feb  7 20:11:23 (none) kernel: [Firmware Bug]: TSC ADJUST differs: CPU0 0 --> -720144233. Restoring
  Feb  7 20:11:23 (none) kernel: microcode: microcode updated early to revision 0xf0, date = 2021-11-12
  Feb  7 20:11:23 (none) kernel: ACPI: PM: Low-level resume complete

I wonder whether anything can be done about that.

Thank you.

--steffen
|
|Der Kragenbaer,                The moon bear,
|der holt sich munter           he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Re: question on microcode loading

[RuiRui Yang]

Ccing x86 and more people.

On Wed, 8 Feb 2023 at 06:09, Steffen Nurpmeso <steffen@sdaoden.eu> wrote:
>
> Hello!
>
> First a thank you for kexec, i use it for a homegrown boot
> solution, where stage1 EFI_STUB kernel is used with busybox and
> cryptsetup to unlock an encrypted device, and then boots into
> stage2 via kexec -- a wonderful and easy solution that makes me
> happy (for years)!
>
> A bit lengthy now.
> There is only one question regarding CPU microcode loading: it
> seems the microcode is not "updated early" with kexec upon "boot".
> This is a homegrown kernel, with built-in firmware
>
>   CONFIG_EXTRA_FIRMWARE="intel-ucode/06-8e-0a ..."
>
> but also with the firmware (prefix)in(g) the initrd
>
>     {
>             # Microcode update must be uncompressed and first
>             [ -f ../../boot/early-ucode.cpio ] && ./$BB cat ../../boot/early-ucode.cpio
>             # Followed by (possibly compressed) normal initrd
>             ./$BB find . | ./$BB cpio -H newc -o | ./$BB gzip -9 -n
>     } > ../.initrd
>
> (ie early-ucode.cpio exists) like here:
>
>   #?0|kent:/boot# cpio -t <.kent.initrd.0
>   kernel
>   kernel/x86
>   kernel/x86/microcode
>   kernel/x86/microcode/.enuineIntel.align.0123456789abc
>   kernel/x86/microcode/GenuineIntel.bin
>   11020 blocks
>   #?0|kent:/boot# dd if=.kent.initrd.0 skip=5642240 bs=1|gunzip|cpio -t
>   .
>   init
>   linux-init-s1.sh
>   sys
>   run
>   proc
>   mnt
>   dev
>   dev/console
>   bin
>   bin/sh
>   etc
>   etc/mdev.sh
>   etc/mdev.conf
>   linux-init-lib.sh
>   linux-init-s2.sh
>   cryptsetup.static
>   busybox.static
>   4491733+0 records in
>   4491733+0 records out
>   4491733 bytes (4.5 MB, 4.3 MiB) copied, 5.67558 s, 791 kB/s
>   18439 blocks
>
> If stage2 then boots it goes like
>
>   Feb  6 17:38:15 (none) kernel: Linux version 6.1.9-ideapad (ports@kent) (gcc (CRUX-x86_64-multilib) 12.2.0, GNU ld (GNU Binutils) 2.39) #1 SMP PREEMPT_DYNAMIC Sat Feb  4 20:40:06 CET 2023
>   Feb  6 17:38:15 (none) kernel: Command line: rtw88_pci.disable_aspm=1 rc.hostname=kent
>   Feb  6 17:38:15 (none) kernel: KERNEL supported cpus:
>   Feb  6 17:38:15 (none) kernel:   Intel GenuineIntel
>   ...
>   Feb  6 17:38:15 (none) kernel: Freeing SMP alternatives memory: 48K
>   Feb  6 17:38:15 (none) kernel: smpboot: CPU0: Intel(R) Core(TM) i5-8250U CPU @ 1.60GHz (family: 0x6, model: 0x8e, stepping: 0xa)
>   ..
>   Feb  6 17:38:15 (none) kernel: .... node  #0, CPUs:      #1 #2 #3 #4
>   Feb  6 17:38:15 (none) kernel: MDS CPU bug present and SMT on, data leak possible. See[..]
>   Feb  6 17:38:15 (none) kernel: MMIO Stale Data CPU bug present and SMT on, data leak possible. See[..]
>   Feb  6 17:38:15 (none) kernel:  #5 #6 #7
>   Feb  6 17:38:15 (none) kernel: smp: Brought up 1 node, 8 CPUs
>   Feb  6 17:38:15 (none) kernel: smpboot: Max logical packages: 1
>   Feb  6 17:38:15 (none) kernel: smpboot: Total of 8 processors activated (28811.00 BogoMIPS)
>   ...
>   Feb  6 17:38:15 (none) kernel: Unpacking initramfs...
>   ...
>   Feb  6 17:38:15 (none) kernel: Freeing initrd memory: 9900K
>   ...
>   Feb  6 17:38:16 (none) kernel: microcode: sig=0x806ea, pf=0x80, revision=0xf0
>   Feb  6 17:38:16 (none) kernel: microcode: Microcode Update Driver: v2.2.
>
> Ie microcode comes late (SMT is later disabled)  It (seems) ok
> after suspend/resume (due to
>   Feb  7 01:30:12 (none) /root/bin/zzz.sh:
>     echo mem > /sys/power/state
>   Feb  7 20:11:26 (none) /root/bin/zzz.sh:
>     echo off > /sys/devices/system/cpu/smt/control
> ):
>
>   Feb  7 01:30:12 (none) kernel: PM: suspend entry (deep)
>   ...
>   Feb  7 20:11:23 (none) kernel: ACPI: PM: Preparing to enter system sleep state S3
>   Feb  7 20:11:23 (none) kernel: ACPI: EC: event blocked
>   Feb  7 20:11:23 (none) kernel: ACPI: EC: EC stopped
>   Feb  7 20:11:23 (none) kernel: ACPI: PM: Saving platform NVS memory
>   Feb  7 20:11:23 (none) kernel: Disabling non-boot CPUs ...
>   Feb  7 20:11:23 (none) kernel: smpboot: CPU 1 is now offline
>   Feb  7 20:11:23 (none) kernel: smpboot: CPU 2 is now offline
>   Feb  7 20:11:23 (none) kernel: smpboot: CPU 3 is now offline
>   Feb  7 20:11:23 (none) kernel: [Firmware Bug]: TSC ADJUST differs: CPU0 0 --> -720144233. Restoring
>   Feb  7 20:11:23 (none) kernel: microcode: microcode updated early to revision 0xf0, date = 2021-11-12
>   Feb  7 20:11:23 (none) kernel: ACPI: PM: Low-level resume complete
>
> I wonder whether anything can be done about that.
>
> Thank you.
>
> --steffen
> |
> |Der Kragenbaer,                The moon bear,
> |der holt sich munter           he cheerfully and one by one
> |einen nach dem anderen runter  wa.ks himself off
> |(By Robert Gernhardt)
>
> _______________________________________________
> kexec mailing list
> kexec@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/kexec
>


_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Re: question on microcode loading

[Borislav Petkov]

On Wed, Feb 08, 2023 at 11:21:13AM +0800, RuiRui Yang wrote:
> > There is only one question regarding CPU microcode loading: it
> > seems the microcode is not "updated early" with kexec upon "boot".

You lost me here: when you load a kernel and you have builtin microcode
or have supplied it through initrd, it should update.

> > This is a homegrown kernel, with built-in firmware

Yeah, this is kinda confusing. When you kexec a kernel, it should be
just the same as when you boot a kernel normally.

If you can give simplified reproduction instructions, I could give it a
try...

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Re: question on microcode loading

[Steffen Nurpmeso]

Borislav Petkov wrote in
 <Y+N4qE2CD9Vu6maw@zn.tnic>:
 |On Wed, Feb 08, 2023 at 11:21:13AM +0800, RuiRui Yang wrote:
 |>> There is only one question regarding CPU microcode loading: it
 |>> seems the microcode is not "updated early" with kexec upon "boot".
 |
 |You lost me here: when you load a kernel and you have builtin microcode
 |or have supplied it through initrd, it should update.

Ok.  That was what i always hoped.  I have never seen "microcode
updated early" for kexec-booted kernels.

(For all CPUs i have it does not make a difference regarding
/sys/devices/system/cpu/vulnerabilities/*, only SMT on/off changes
things; ie, CPU microcode is not newest but seems to offer
mitigations already; and likewise new microcode does not mitigate
the remaining vulnerabilities; and it unfortunately seems FINEIBT
is not backported to 6.1, but that off-topic.)

I only wanted to ask once, as the message comes so late, when all
CPUs are in use.  (And the "updated early" _does_ come after
suspend/resume.  And i am no kernel / hardware hacker.)

 |>> This is a homegrown kernel, with built-in firmware
 |
 |Yeah, this is kinda confusing. When you kexec a kernel, it should be
 |just the same as when you boot a k^ernel normally.

Very lucky for that kexec possibility, thanks.
I have not seen the "normal" case for long, at least not the log
(it goes nowhere), and the screen scrolls too fast.  (And there is
no scrollback buffer, and i think i am 

 |If you can give simplified reproduction instructions, I could give it a
 |try...

Well it is a simple kernel with enough built-in to boot stage1 so
i can run cryptsetup, and then (optionally) kexec-boot stage2
(same (maybe newer) kernel but on encrypted device).  Formerly all
static, but now network/firewall, sound etc are modules.
Three small somewhat simple scripts with text-only config files.
But upon interest i can very well post them?  (I like it!  No
extra boot loader, no systemd here.)

Thank you.  And ciao!

--steffen
|
|Der Kragenbaer,                The moon bear,
|der holt sich munter           he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec