On Mon, 16 Jan 2017, Roger Pau Monné wrote: > On Mon, Jan 16, 2017 at 09:50:53AM -0700, Jan Beulich wrote: > > >>> On 16.01.17 at 17:31, wrote: > > > On Mon, Jan 16, 2017 at 09:09:55AM -0700, Jan Beulich wrote: > > >> >>> On 16.01.17 at 16:14, wrote: > > >> > On Fri, Jan 13, 2017 at 08:51:57AM -0700, Jan Beulich wrote: > > >> >> >>> On 12.01.17 at 13:13, wrote: > > >> >> > # Introduction > > >> >> > > > >> >> > One of the design goals of PVH is to be able to remove as much Xen PV specific > > >> >> > code as possible, thus limiting the number of Xen PV interfaces used by guests, > > >> >> > and tending to use native interfaces (as used by bare metal) as much as > > >> >> > possible. This is in line with the efforts also done by Xen on ARM and helps > > >> >> > reduce the burden of maintaining huge amounts of Xen PV code inside of guests > > >> >> > kernels. > > >> >> > > > >> >> > This however presents some challenges due to the model used by the Xen > > >> >> > Hypervisor, where some devices are handled by Xen while others are left for the > > >> >> > hardware domain to manage. The fact that Xen lacks and AML parser also makes it > > >> >> > harder, since it cannot get the full hardware description from dynamic ACPI > > >> >> > tables (DSDT, SSDT) without the hardware domain collaboration. > > >> >> > > >> >> Considering all the difficulties with the proposed model plus the little > > >> >> code the PV vCPU hotplug logic requires in the kernel (assuming a > > >> >> xenbus driver to be there anyway), I'm rather unconvinced that > > >> >> going this route instead of sticking to the PV model is actually > > >> >> desirable. And clearly, for consistency within the kernel, in such a > > >> >> case I'd then also favor sticking to this model for DomU. > > >> > > > >> > We would at least have to pass the APIC ID in order to perform vCPU hotplug for > > >> > PVH, and the ACPI spec mandates that when using x2APIC structures in the MADT, > > >> > there must be a matching processor object in the DSDT (5.2.12.12). > > >> > > > >> > Declaring processor objects in the DSDT won't be possible for Xen, but we can > > >> > at least declare them in a SSDT, which seems better than not doing it at all. > > >> > Maybe we can get ACPI to loosen the spec and don't mandate DSDT anymore. > > >> > > >> I don't understand this reply of yours: How do any ACPI requirements > > >> come into play when using the PV hotplug mechanism for vCPU-s? > > > > > > This clearly isn't a requirement when doing PV vCPU hotplug, but it's a > > > violation of the spec (proving x2APIC entries without matching processor > > > objects), so I wouldn't be surprised if ACPICA or any other ACPI implementation > > > refuses to boot on systems with x2APIC entries but no processor objects. > > > > Good point, but what do you suggest short of declaring PVH v2 Dom0 > > impossible to properly implement? I think that the idea of multiplexing > > ACPI for different purposes is simply going too far. For PV there's no > > such problem, as the Dom0 OS is expected to be aware that processor > > information coming from ACPI is not applicable to the view on CPUs it > > has (read: vCPU-s). And therefore, unless clean multiplexing is possible, > > I think PVH will need to retain this requirement (at which point there's > > no spec violation anymore). > > But we definitely want to use ACPI to pass the boot vCPU information, using the > MADT for both DomU and Dom0. > > Then for PVH DomU using ACPI vCPU hotplug makes perfect sense, it requires less > Xen specific code in the OS and it's fairly easy to implement inside of > Xen/toolstack. But I understand that using different methods for DomU vs Dom0 > is very awkward. I still think that ACPI vCPU hotplug for Dom0 this is not so > far-fetched, and that it could be doable. > > Could we introduce a new CPUID flag to notify the guest of whether it should > expect ACPI vCPU hotplug or PV vCPU hotplug? This is a good idea. > I don't really like having Xen-specific checks inside of OSes, like "it's PVH > guest" then short circuiting a bunch of native logic. For example the ACPICA > ACPI shutdown hooks for Xen Dom0 never made it upstream, and it's very hard for > me to argue with the FreeBSD ACPICA maintainer about why those are needed, > and why he has to maintain a patch on top of upstream ACPICA only for Xen. Nobody likes those. However, I don't think that PV cpu hotplug requires any of those.