Re: [RFC Patch v4 4/8] hvmloader: boot cpu through broadcast

From: "Roger Pau Monné" <roger.pau@citrix.com>
To: Chao Gao <chao.gao@intel.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
	Wei Liu <wei.liu2@citrix.com>,
	Ian Jackson <ian.jackson@eu.citrix.com>,
	Jan Beulich <JBeulich@suse.com>,
	xen-devel@lists.xen.org
Subject: Re: [RFC Patch v4 4/8] hvmloader: boot cpu through broadcast
Date: Mon, 26 Feb 2018 14:19:10 +0000	[thread overview]
Message-ID: <20180226141910.iy3zjgajwyizkm6o@MacBook-Pro-de-Roger.local> (raw)
In-Reply-To: <20180226123322.GA140947@skl-4s-chao.sh.intel.com>

On Mon, Feb 26, 2018 at 08:33:23PM +0800, Chao Gao wrote:
> On Mon, Feb 26, 2018 at 01:28:07AM -0700, Jan Beulich wrote:
> >>>> On 24.02.18 at 06:49, <chao.gao@intel.com> wrote:
> >> On Fri, Feb 23, 2018 at 04:42:10PM +0000, Roger Pau Monné wrote:
> >>>On Wed, Dec 06, 2017 at 03:50:10PM +0800, Chao Gao wrote:
> >>>> Intel SDM Extended XAPIC (X2APIC) -> "Initialization by System Software"
> >>>> has the following description:
> >>>> 
> >>>> "The ACPI interfaces for the x2APIC are described in Section 5.2, “ACPI System
> >>>> Description Tables,” of the Advanced Configuration and Power Interface
> >>>> Specification, Revision 4.0a (http://www.acpi.info/spec.htm). The default
> >>>> behavior for BIOS is to pass the control to the operating system with the
> >>>> local x2APICs in xAPIC mode if all APIC IDs reported by CPUID.0BH:EDX are less
> >>>> than 255, and in x2APIC mode if there are any logical processor reporting an
> >>>> APIC ID of 255 or greater."
> >>>> 
> >>>> In this patch, hvmloader enables x2apic mode for all vcpus if there are cpus
> >>>> with APIC ID > 255. To wake up processors whose APIC ID is greater than 255,
> >>>> the SIPI is broadcasted to all APs. It is the way how Seabios wakes up APs.
> >>>> APs may compete for the stack, thus a lock is introduced to protect the stack.
> >>>
> >>>Hm, how are we going to deal with this on PVH? hvmloader doesn't run
> >>>for PVH guests, hence it seems like switching to x2APIC mode should be
> >>>done somewhere else that shared between HVM and PVH.
> >>>
> >>>Maybe the hypercall that sets the number of vCPUs should change the
> >>>APIC mode?
> >> 
> >> Yes. A flag can be passed when setting the maximum number of vCPUs. Xen
> >> will switch all local APICs to x2APIC mode or xAPIC mode according to
> >> the flag.
> >
> >A flag? Where? Why isn't 257+ vCPU-s on its own sufficient to tell
> >that the mode needs to be switched?
> 
> In struct xen_domctl_max_vcpus, a flag, like SWITCH_TO_X2APIC_MODE, can
> be used to instruct Xen to initialize vlapic and do this switch.
> 
> Yes, it is another option: Xen can do this switch when need. This
> solution leads to smaller code change compared with introducing a new
> flag when setting the maximum number of vCPUs.

Since APIC ID is currently hardcoded in guest_cpuid as vcpu_id * 2,
IMO Xen should switch to x2APIC mode when it detects that vCPUs > 128,
like Jan has suggest. Then you won't need to modify hvmloader at all,
and the same would work for PVH I assume?

Thanks, Roger.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel