Re: PVH CPU hotplug design document

["Jan Beulich" <JBeulich@suse.com>]

>>> On 12.01.17 at 20:00, <andrew.cooper3@citrix.com> wrote:
> On 12/01/17 12:13, Roger Pau Monné wrote:
>> ## Proposed solution using the STAO
>>
>> The general idea of this method is to use the STAO in order to hide the pCPUs
>> from the hardware domain, and provide processor objects for vCPUs in an extra
>> SSDT table.
>>
>> This method requires one change to the STAO, in order to be able to notify the
>> hardware domain of which processors found in ACPI tables are pCPUs. The
>> description of the new STAO field is as follows:
>>
>>  |   Field            | Byte Length | Byte Offset |     Description          |
>>  |--------------------|:-----------:|:-----------:|--------------------------|
>>  | Processor List [n] |      -      |      -      | A list of ACPI numbers,  |
>>  |                    |             |             | where each number is the |
>>  |                    |             |             | Processor UID of a       |
>>  |                    |             |             | physical CPU, and should |
>>  |                    |             |             | be treated specially by  |
>>  |                    |             |             | the OSPM                 |
>>
>> The list of UIDs in this new field would be matched against the ACPI Processor
>> UID field found in local/x2 APIC MADT structs and Processor objects in the ACPI
>> namespace, and the OSPM should either ignore those objects, or in case it
>> implements pCPU hotplug, it should notify Xen of changes to these objects.
>>
>> The contents of the MADT provided to the hardware domain are also going to be
>> different from the contents of the MADT as found in native ACPI. The local/x2
>> APIC entries for all the pCPUs are going to be marked as disabled.
>>
>> Extra entries are going to be added for each vCPU available to the hardware
>> domain, up to the maximum number of supported vCPUs. Note that supported vCPUs
>> might be different than enabled vCPUs, so it's possible that some of these
>> entries are also going to be marked as disabled. The entries for vCPUs on the
>> MADT are going to use a processor local x2 APIC structure, and the ACPI
>> processor ID of the first vCPU is going to be UINT32_MAX - HVM_MAX_VCPUS, in
>> order to avoid clashes with IDs of pCPUs.
> 
> This is slightly problematic.  There is no restriction (so far as I am
> aware) on which ACPI IDs the firmware picks for its objects.  They need
> not be consecutive, logical, or start from 0.
> 
> If STAO is being extended to list the IDs of the physical processor
> objects, we should go one step further and explicitly list the IDs of
> the virtual processor objects.  This leaves us flexibility if we have to
> avoid awkward firmware ID layouts.

I don't think we should do this - vCPU IDs are already in MADT. I do,
however, think that we shouldn't name any specific IDs we mean to
use for the vCPU-s, but rather merely guarantee that there won't be
any overlap with the pCPU ones.

>> In order to be able to perform vCPU hotplug, the vCPUs must have an ACPI
>> processor object in the ACPI namespace, so that the OSPM can request
>> notifications and get the value of the \_STA and \_MAT methods. This can be
>> problematic because Xen doesn't know the ACPI name of the other processor
>> objects, so blindly adding new ones can create namespace clashes.
>>
>> This can be solved by using a different ACPI name in order to describe vCPUs in
>> the ACPI namespace. Most hardware vendors tend to use CPU or PR prefixes for
>> the processor objects, so using a 'VP' (ie: Virtual Processor) prefix should
>> prevent clashes.
> 
> One system I have to hand (with more than 255 pcpus) uses Cxxx
> 
> To avoid namespace collisions, I can't see any option but to parse the
> DSDT/SSDTs to at least confirm that VPxx is available to use.

And additionally using a two character name prefix would significantly
limit the number of vCPU-s we would be able to support going forward.
Just like above, I don't think we should specify the name here at all,
allowing dynamic picking of suitable ones.

>> A Xen GPE device block will be used in order to deliver events related to the
>> vCPUs available to the guest, since Xen doesn't know if there are any bits
>> available in the native GPEs. A SCI interrupt will be injected into the guest
>> in order to trigger the event.
>>
>> The following snippet is a representation of the ASL SSDT code that is proposed
>> for the hardware domain:
>>
>>     DefinitionBlock ("SSDT.aml", "SSDT", 5, "Xen", "HVM", 0)
>>     {
>>         Scope (\_SB)
>>         {
>>            OperationRegion(XEN, SystemMemory, 0xDEADBEEF, 40)
>>            Field(XEN, ByteAcc, NoLock, Preserve) {
>>                NCPU, 16, /* Number of vCPUs */
>>                MSUA, 32, /* MADT checksum address */
>>                MAPA, 32, /* MADT LAPIC0 address */
>>            }
[...]
>> Since the position of the XEN data memory area is not know, the hypervisor will
>> have to replace the address 0xdeadbeef with the actual memory address where
>> this structure has been copied. This will involve a memory search of the AML
>> code resulting from the compilation of the above ASL snippet.
> 
> This is also slightly risky.  If we need to do this, can we get a
> relocation list from the compiled table from iasl?

I expect iasl can't do that, the more that there's not actually any
relocation involved here. I guess we'd need a double compilation
approach, where for both a different address is being specified.
The diff of the two would then allow to create a relocation list.

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel