qemu-devel.nongnu.org archive mirror
 help / color / mirror / Atom feed
From: "Yao, Jiewen" <jiewen.yao@intel.com>
To: Paolo Bonzini <pbonzini@redhat.com>,
	Laszlo Ersek <lersek@redhat.com>,
	edk2-devel-groups-io <devel@edk2.groups.io>
Cc: "Chen, Yingwen" <yingwen.chen@intel.com>,
	Phillip Goerl <phillip.goerl@oracle.com>,
	qemu devel list <qemu-devel@nongnu.org>,
	"Nakajima, Jun" <jun.nakajima@intel.com>,
	Igor Mammedov <imammedo@redhat.com>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	edk2-rfc-groups-io <rfc@edk2.groups.io>,
	Joao Marcal Lemos Martins <joao.m.martins@oracle.com>
Subject: Re: [Qemu-devel] CPU hotplug using SMM with QEMU+OVMF
Date: Thu, 15 Aug 2019 09:55:11 +0000	[thread overview]
Message-ID: <74D8A39837DF1E4DA445A8C0B3885C503F75E4E9@shsmsx102.ccr.corp.intel.com> (raw)
In-Reply-To: <047801f8-624a-2300-3cf7-1daa1395ce59@redhat.com>

Hi Paolo
I am not sure what do you mean - "You do not need a reset vector ...".
If so, where is the first instruction of the new CPU in the virtualization environment?
Please help me understand that at first. Then we can continue the discussion.

Thank you
Yao Jiewen

> -----Original Message-----
> From: Paolo Bonzini [mailto:pbonzini@redhat.com]
> Sent: Wednesday, August 14, 2019 10:05 PM
> To: Yao, Jiewen <jiewen.yao@intel.com>; Laszlo Ersek
> <lersek@redhat.com>; edk2-devel-groups-io <devel@edk2.groups.io>
> Cc: edk2-rfc-groups-io <rfc@edk2.groups.io>; qemu devel list
> <qemu-devel@nongnu.org>; Igor Mammedov <imammedo@redhat.com>;
> Chen, Yingwen <yingwen.chen@intel.com>; Nakajima, Jun
> <jun.nakajima@intel.com>; Boris Ostrovsky <boris.ostrovsky@oracle.com>;
> Joao Marcal Lemos Martins <joao.m.martins@oracle.com>; Phillip Goerl
> <phillip.goerl@oracle.com>
> Subject: Re: CPU hotplug using SMM with QEMU+OVMF
> 
> On 14/08/19 15:20, Yao, Jiewen wrote:
> >> - Does this part require a new branch somewhere in the OVMF SEC code?
> >>   How do we determine whether the CPU executing SEC is BSP or
> >>   hot-plugged AP?
> > [Jiewen] I think this is blocked from hardware perspective, since the first
> instruction.
> > There are some hardware specific registers can be used to determine if the
> CPU is new added.
> > I don’t think this must be same as the real hardware.
> > You are free to invent some registers in device model to be used in OVMF
> hot plug driver.
> 
> Yes, this would be a new operation mode for QEMU, that only applies to
> hot-plugged CPUs.  In this mode the AP doesn't reply to INIT or SMI, in
> fact it doesn't reply to anything at all.
> 
> >> - How do we tell the hot-plugged AP where to start execution? (I.e. that
> >>   it should execute code at a particular pflash location.)
> > [Jiewen] Same real mode reset vector at FFFF:FFF0.
> 
> You do not need a reset vector or INIT/SIPI/SIPI sequence at all in
> QEMU.  The AP does not start execution at all when it is unplugged, so
> no cache-as-RAM etc.

> We only need to modify QEMU so that hot-plugged APIs do not reply to
> INIT/SIPI/SMI.
> 
> > I don’t think there is problem for real hardware, who always has CAR.
> > Can QEMU provide some CPU specific space, such as MMIO region?
> 
> Why is a CPU-specific region needed if every other processor is in SMM
> and thus trusted.
> >>   Does CPU hotplug apply only at the socket level? If the CPU is
> >>   multi-core, what is responsible for hot-plugging all cores present in
> >>   the socket?
> 
> I can answer this: the SMM handler would interact with the hotplug
> controller in the same way that ACPI DSDT does normally.  This supports
> multiple hotplugs already.
> 
> Writes to the hotplug controller from outside SMM would be ignored.
> 
> >>> (03) New CPU: (Flash) send board message to tell host CPU (GPIO->SCI)
> >>>      -- I am waiting for hot-add message.
> >>
> >> Maybe we can simplify this in QEMU by broadcasting an SMI to existent
> >> processors immediately upon plugging the new CPU.
> 
> The QEMU DSDT could be modified (when secure boot is in effect) to OUT
> to 0xB2 when hotplug happens.  It could write a well-known value to
> 0xB2, to be read by an SMI handler in edk2.
> 
> 
> >>
> >>>                                        (NOTE: Host CPU can
> only
> >> send
> >>>      instruction in SMM mode. -- The register is SMM only)
> >>
> >> Sorry, I don't follow -- what register are we talking about here, and
> >> why is the BSP needed to send anything at all? What "instruction" do you
> >> have in mind?
> > [Jiewen] The new CPU does not enable SMI at reset.
> > At some point of time later, the CPU need enable SMI, right?
> > The "instruction" here means, the host CPUs need tell to CPU to enable
> SMI.
> 
> Right, this would be a write to the CPU hotplug controller
> 
> >>> (04) Host CPU: (OS) get message from board that a new CPU is added.
> >>>      (GPIO -> SCI)
> >>>
> >>> (05) Host CPU: (OS) All CPUs enter SMM (SCI->SWSMI) (NOTE: New CPU
> >>>      will not enter CPU because SMI is disabled)
> >>
> >> I don't understand the OS involvement here. But, again, perhaps QEMU
> can
> >> force all existent CPUs into SMM immediately upon adding the new CPU.
> > [Jiewen] OS here means the Host CPU running code in OS environment, not
> in SMM environment.
> 
> See above.
> 
> >>> (06) Host CPU: (SMM) Save 38000, Update 38000 -- fill simple SMM
> >>>      rebase code.
> >>>
> >>> (07) Host CPU: (SMM) Send message to New CPU to Enable SMI.
> >>
> >> Aha, so this is the SMM-only register you mention in step (03). Is the
> >> register specified in the Intel SDM?
> > [Jiewen] Right. That is the register to let host CPU tell new CPU to enable
> SMI.
> > It is platform specific register. Not defined in SDM.
> > You may invent one in device model.
> 
> See above.
> 
> >>> (10) New CPU: (SMM) Response first SMI at 38000, and rebase SMBASE
> to
> >>>      TSEG.
> >>
> >> What code does the new CPU execute after it completes step (10)? Does
> it
> >> halt?
> >
> > [Jiewen] The new CPU exits SMM and return to original place - where it is
> > interrupted to enter SMM - running code on the flash.
> 
> So in our case we'd need an INIT/SIPI/SIPI sequence between (06) and (07).
> 
> >>> (11) Host CPU: (SMM) Restore 38000.
> >>
> >> These steps (i.e., (06) through (11)) don't appear RAS-specific. The
> >> only platform-specific feature seems to be SMI masking register, which
> >> could be extracted into a new SmmCpuFeaturesLib API.
> >>
> >> Thus, would you please consider open sourcing firmware code for steps
> >> (06) through (11)?
> >>
> >> Alternatively -- and in particular because the stack for step (01)
> >> concerns me --, we could approach this from a high-level, functional
> >> perspective. The states that really matter are the relocated SMBASE for
> >> the new CPU, and the state of the full system, right at the end of step
> >> (11).
> >>
> >> When the SMM setup quiesces during normal firmware boot, OVMF could
> >> use
> >> existent (finalized) SMBASE infomation to *pre-program* some virtual
> >> QEMU hardware, with such state that would be expected, as "final" state,
> >> of any new hotplugged CPU. Afterwards, if / when the hotplug actually
> >> happens, QEMU could blanket-apply this state to the new CPU, and
> >> broadcast a hardware SMI to all CPUs except the new one.
> 
> I'd rather avoid this and stay as close as possible to real hardware.
> 
> Paolo

  reply	other threads:[~2019-08-15  9:56 UTC|newest]

Thread overview: 66+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-08-13 14:16 [Qemu-devel] CPU hotplug using SMM with QEMU+OVMF Laszlo Ersek
2019-08-13 16:09 ` Laszlo Ersek
2019-08-13 16:18   ` Laszlo Ersek
2019-08-14 13:20   ` Yao, Jiewen
2019-08-14 14:04     ` Paolo Bonzini
2019-08-15  9:55       ` Yao, Jiewen [this message]
2019-08-15 16:04         ` Paolo Bonzini
2019-08-15 15:00       ` [Qemu-devel] [edk2-devel] " Laszlo Ersek
2019-08-15 16:16         ` Igor Mammedov
2019-08-15 16:21         ` Paolo Bonzini
2019-08-16  2:46           ` Yao, Jiewen
2019-08-16  7:20             ` Paolo Bonzini
2019-08-16  7:49               ` Yao, Jiewen
2019-08-16 20:15                 ` Laszlo Ersek
2019-08-16 22:19                   ` Alex Williamson
2019-08-17  0:20                     ` Yao, Jiewen
2019-08-18 19:50                       ` Paolo Bonzini
2019-08-18 23:00                         ` Yao, Jiewen
2019-08-19 14:10                           ` Paolo Bonzini
2019-08-21 12:07                             ` Laszlo Ersek
     [not found]                           ` <E92EE9817A31E24EB0585FDF735412F5B9D9C671@ORSMSX113.amr.corp.intel.com>
2019-08-21 17:05                             ` [Qemu-devel] [edk2-rfc] " Paolo Bonzini
     [not found]                               ` <E92EE9817A31E24EB0585FDF735412F5B9D9D74A@ORSMSX113.amr.corp.intel.com>
2019-08-21 17:39                                 ` Paolo Bonzini
2019-08-21 20:17                                   ` Kinney, Michael D
2019-08-22  6:18                                     ` Paolo Bonzini
2019-08-22 18:29                                       ` Laszlo Ersek
2019-08-22 18:51                                         ` Paolo Bonzini
2019-08-23 14:53                                           ` Laszlo Ersek
2019-08-22 20:13                                         ` Kinney, Michael D
2019-08-22 17:59                               ` Laszlo Ersek
2019-08-22 18:43                                 ` Paolo Bonzini
2019-08-22 20:06                                   ` Kinney, Michael D
2019-08-22 22:18                                     ` Paolo Bonzini
2019-08-22 22:32                                       ` Kinney, Michael D
2019-08-22 23:11                                         ` Paolo Bonzini
2019-08-23  1:02                                           ` Kinney, Michael D
2019-08-23  5:00                                             ` Yao, Jiewen
2019-08-23 15:25                                               ` Kinney, Michael D
2019-08-24  1:48                                                 ` Yao, Jiewen
2019-08-27 18:31                                                   ` Igor Mammedov
2019-08-29 17:01                                                     ` Laszlo Ersek
2019-08-30 14:48                                                       ` Igor Mammedov
2019-08-30 18:46                                                         ` Laszlo Ersek
2019-09-02  8:45                                                           ` Igor Mammedov
2019-09-02 19:09                                                             ` Laszlo Ersek
2019-09-03 14:53                                                               ` Igor Mammedov
2019-09-03 17:20                                                                 ` Laszlo Ersek
2019-09-04  9:52                                                                   ` Igor Mammedov
2019-09-05 13:08                                                                     ` Laszlo Ersek
2019-09-05 15:45                                                                       ` Igor Mammedov
2019-09-05 15:49                                                                       ` [Qemu-devel] [PATCH] q35: lpc: allow to lock down 128K RAM at default SMBASE address Igor Mammedov
2019-09-09 19:15                                                                         ` Laszlo Ersek
2019-09-09 19:20                                                                           ` Laszlo Ersek
2019-09-10 15:58                                                                           ` Igor Mammedov
2019-09-11 17:30                                                                             ` Laszlo Ersek
2019-09-17 13:11                                                                               ` [Qemu-devel] [edk2-devel] " Igor Mammedov
2019-08-26 15:30                                                 ` [Qemu-devel] [edk2-rfc] [edk2-devel] CPU hotplug using SMM with QEMU+OVMF Laszlo Ersek
2019-08-27 16:23                                                   ` Igor Mammedov
2019-08-27 20:11                                                     ` Laszlo Ersek
2019-08-28 12:01                                                       ` Igor Mammedov
2019-08-29 16:25                                                         ` Laszlo Ersek
2019-08-30 13:49                                                           ` Igor Mammedov
2019-08-22 17:53                             ` Laszlo Ersek
2019-08-16 20:00           ` [Qemu-devel] " Laszlo Ersek
2019-08-15 16:07       ` [Qemu-devel] " Igor Mammedov
2019-08-15 16:24         ` Paolo Bonzini
2019-08-16  7:42           ` Igor Mammedov

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=74D8A39837DF1E4DA445A8C0B3885C503F75E4E9@shsmsx102.ccr.corp.intel.com \
    --to=jiewen.yao@intel.com \
    --cc=boris.ostrovsky@oracle.com \
    --cc=devel@edk2.groups.io \
    --cc=imammedo@redhat.com \
    --cc=joao.m.martins@oracle.com \
    --cc=jun.nakajima@intel.com \
    --cc=lersek@redhat.com \
    --cc=pbonzini@redhat.com \
    --cc=phillip.goerl@oracle.com \
    --cc=qemu-devel@nongnu.org \
    --cc=rfc@edk2.groups.io \
    --cc=yingwen.chen@intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).