PVH and ACPI discussion

PVH and ACPI discussion

[Roger Pau Monné]

Hello,

While looking at some tangential issues I realized that the 'VGA Not
Present' flag that Xen currently sets for PVH DomUs might be slightly
different from what we expect it to mean. The purpose was that Xen
would set this flag to denote there's no VGA MMIO region in the low
1MB, so that the guest OS would not reserve memory in that area
thinking there's a MMIO window there. The memory map provided to a PVH
DomU typically contains a single RAM range that expands from 0 to the
selected amount of memory.

The description of such flag by the ACPI spec (6.2A) however is as
follows:

"If set, indicates to OSPM that it must not blindly probe the VGA
hardware (that responds to MMIO addresses A0000h-BFFFFh and IO ports
3B0h-3BBh and 3C0h-3DFh) that may cause machine check on this system.
If clear, indicates to OSPM that it is safe to probe the VGA
hardware."

My reading of the above text would make me think that if the flag is
set the memory region A0000h-BFFFFh should not be used at all, and
that would be in conflict with the memory map that's provided to
guests (which lists this area as RAM).

I'm not convinced of the best way to proceed here. I can contact the
ACPI working group and try to clarify the meaning of the flag, or
inquiry if there's a more suitable flag for or use case, but I would
like to hear others opinion on this topic.

Secondly, I've also been looking at whether it would make sense to set
the ACPI reduced hardware flag for PVH DomUs, according to the
description in the ACPI spec:

"For certain classes of systems the ACPI Hardware Specification may
not be adequate. Examples include legacy-free, UEFI-based platforms
with recent processors, and those implementing mobile platform
architectures. For such platforms, a Hardware-reduced ACPI mode is
defined."

Certainly the legacy-free and UEFI part is quite applicable to PVH
DomU, for which we don't plan to support legacy BIOS and only provide
UEFI firmware in the long term.

Reduced HW ACPI also gets rid of the SCI interrupt, and instead
provides some other methods to signal ACPI events (note we don't
use any ACPI event ATM for PVH DomU). It also gets rid of a bunch of
FADT fields that we don't use for PVH DomU either.

I however seem to remember some past discussion about PVH DomU and
reduced ACPI, but I cannot find the thread. If there are no objections
I think we should look into this (likely discuss with the ACPI working
group) in order to figure out if reduced HW ACPI could work for us,
and how the event delivery could be implemented for PVH DomU if it
turns out we need it later on. It might make sense to also figure out
what other people do, like HyperV Gen2 instances (which also get rid
of a lot of legacy hw).

Thanks, Roger.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: PVH and ACPI discussion

[Boris Ostrovsky]

On 1/3/19 12:45 PM, Roger Pau Monné wrote:
> Hello,
>
> While looking at some tangential issues I realized that the 'VGA Not
> Present' flag that Xen currently sets for PVH DomUs might be slightly
> different from what we expect it to mean. The purpose was that Xen
> would set this flag to denote there's no VGA MMIO region in the low
> 1MB, so that the guest OS would not reserve memory in that area
> thinking there's a MMIO window there. The memory map provided to a PVH
> DomU typically contains a single RAM range that expands from 0 to the
> selected amount of memory.
>
> The description of such flag by the ACPI spec (6.2A) however is as
> follows:
>
> "If set, indicates to OSPM that it must not blindly probe the VGA
> hardware (that responds to MMIO addresses A0000h-BFFFFh and IO ports
> 3B0h-3BBh and 3C0h-3DFh) that may cause machine check on this system.
> If clear, indicates to OSPM that it is safe to probe the VGA
> hardware."
>
> My reading of the above text would make me think that if the flag is
> set the memory region A0000h-BFFFFh should not be used at all, and
> that would be in conflict with the memory map that's provided to
> guests (which lists this area as RAM).
>
> I'm not convinced of the best way to proceed here. I can contact the
> ACPI working group and try to clarify the meaning of the flag, or
> inquiry if there's a more suitable flag for or use case, but I would
> like to hear others opinion on this topic.
>
> Secondly, I've also been looking at whether it would make sense to set
> the ACPI reduced hardware flag for PVH DomUs, according to the
> description in the ACPI spec:
>
> "For certain classes of systems the ACPI Hardware Specification may
> not be adequate. Examples include legacy-free, UEFI-based platforms
> with recent processors, and those implementing mobile platform
> architectures. For such platforms, a Hardware-reduced ACPI mode is
> defined."
>
> Certainly the legacy-free and UEFI part is quite applicable to PVH
> DomU, for which we don't plan to support legacy BIOS and only provide
> UEFI firmware in the long term.
>
> Reduced HW ACPI also gets rid of the SCI interrupt, and instead
> provides some other methods to signal ACPI events (note we don't
> use any ACPI event ATM for PVH DomU). It also gets rid of a bunch of
> FADT fields that we don't use for PVH DomU either.
>
> I however seem to remember some past discussion about PVH DomU and
> reduced ACPI, but I cannot find the thread. If there are no objections
> I think we should look into this (likely discuss with the ACPI working
> group) in order to figure out if reduced HW ACPI could work for us,
> and how the event delivery could be implemented for PVH DomU if it
> turns out we need it later on. It might make sense to also figure out
> what other people do, like HyperV Gen2 instances (which also get rid
> of a lot of legacy hw).

I found a couple of thread where we discussed this:

https://lists.xenproject.org/archives/html/xen-devel/2016-02/msg01462.html
https://lists.xenproject.org/archives/html/xen-devel/2016-06/msg00675.html

I can't find where we actually rejected it but I think it had something
to with the fact that we can't use it for dom0.

-boris




_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: PVH and ACPI discussion

[Roger Pau Monné]

On Thu, Jan 03, 2019 at 03:42:18PM -0500, Boris Ostrovsky wrote:
> On 1/3/19 12:45 PM, Roger Pau Monné wrote:
> > Hello,
> >
> > While looking at some tangential issues I realized that the 'VGA Not
> > Present' flag that Xen currently sets for PVH DomUs might be slightly
> > different from what we expect it to mean. The purpose was that Xen
> > would set this flag to denote there's no VGA MMIO region in the low
> > 1MB, so that the guest OS would not reserve memory in that area
> > thinking there's a MMIO window there. The memory map provided to a PVH
> > DomU typically contains a single RAM range that expands from 0 to the
> > selected amount of memory.
> >
> > The description of such flag by the ACPI spec (6.2A) however is as
> > follows:
> >
> > "If set, indicates to OSPM that it must not blindly probe the VGA
> > hardware (that responds to MMIO addresses A0000h-BFFFFh and IO ports
> > 3B0h-3BBh and 3C0h-3DFh) that may cause machine check on this system.
> > If clear, indicates to OSPM that it is safe to probe the VGA
> > hardware."
> >
> > My reading of the above text would make me think that if the flag is
> > set the memory region A0000h-BFFFFh should not be used at all, and
> > that would be in conflict with the memory map that's provided to
> > guests (which lists this area as RAM).
> >
> > I'm not convinced of the best way to proceed here. I can contact the
> > ACPI working group and try to clarify the meaning of the flag, or
> > inquiry if there's a more suitable flag for or use case, but I would
> > like to hear others opinion on this topic.
> >
> > Secondly, I've also been looking at whether it would make sense to set
> > the ACPI reduced hardware flag for PVH DomUs, according to the
> > description in the ACPI spec:
> >
> > "For certain classes of systems the ACPI Hardware Specification may
> > not be adequate. Examples include legacy-free, UEFI-based platforms
> > with recent processors, and those implementing mobile platform
> > architectures. For such platforms, a Hardware-reduced ACPI mode is
> > defined."
> >
> > Certainly the legacy-free and UEFI part is quite applicable to PVH
> > DomU, for which we don't plan to support legacy BIOS and only provide
> > UEFI firmware in the long term.
> >
> > Reduced HW ACPI also gets rid of the SCI interrupt, and instead
> > provides some other methods to signal ACPI events (note we don't
> > use any ACPI event ATM for PVH DomU). It also gets rid of a bunch of
> > FADT fields that we don't use for PVH DomU either.
> >
> > I however seem to remember some past discussion about PVH DomU and
> > reduced ACPI, but I cannot find the thread. If there are no objections
> > I think we should look into this (likely discuss with the ACPI working
> > group) in order to figure out if reduced HW ACPI could work for us,
> > and how the event delivery could be implemented for PVH DomU if it
> > turns out we need it later on. It might make sense to also figure out
> > what other people do, like HyperV Gen2 instances (which also get rid
> > of a lot of legacy hw).
> 
> I found a couple of thread where we discussed this:
> 
> https://lists.xenproject.org/archives/html/xen-devel/2016-02/msg01462.html
> https://lists.xenproject.org/archives/html/xen-devel/2016-06/msg00675.html

Thanks!

> I can't find where we actually rejected it but I think it had something
> to with the fact that we can't use it for dom0.

Yes, Dom0 will get the (almost) native ACPI tables, so if the
underlying firmware has set the reduced hw flag Dom0 would inherit
it.

IMO setting the flag for DomUs should be independent of what it's done
for Dom0, since in the Dom0 case Xen doesn't fabricate a full set of
ACPI tables for Dom0, and just mangles the native ones a little bit.

Roger.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: PVH and ACPI discussion

[Jan Beulich]

>>> On 03.01.19 at 18:45, <roger.pau@citrix.com> wrote:
> While looking at some tangential issues I realized that the 'VGA Not
> Present' flag that Xen currently sets for PVH DomUs might be slightly
> different from what we expect it to mean. The purpose was that Xen
> would set this flag to denote there's no VGA MMIO region in the low
> 1MB, so that the guest OS would not reserve memory in that area
> thinking there's a MMIO window there. The memory map provided to a PVH
> DomU typically contains a single RAM range that expands from 0 to the
> selected amount of memory.
> 
> The description of such flag by the ACPI spec (6.2A) however is as
> follows:
> 
> "If set, indicates to OSPM that it must not blindly probe the VGA
> hardware (that responds to MMIO addresses A0000h-BFFFFh and IO ports
> 3B0h-3BBh and 3C0h-3DFh) that may cause machine check on this system.
> If clear, indicates to OSPM that it is safe to probe the VGA
> hardware."
> 
> My reading of the above text would make me think that if the flag is
> set the memory region A0000h-BFFFFh should not be used at all, and
> that would be in conflict with the memory map that's provided to
> guests (which lists this area as RAM).
> 
> I'm not convinced of the best way to proceed here. I can contact the
> ACPI working group and try to clarify the meaning of the flag, or
> inquiry if there's a more suitable flag for or use case, but I would
> like to hear others opinion on this topic.

"Should not blindly probe" != "should not use". To me the wording
implies that some secondary means are necessary to tell what the
region is used for, which is left unspecified by ACPI itself. If that's
what's meant, we're free to make our secondary spec "it's RAM in
all cases".

> Secondly, I've also been looking at whether it would make sense to set
> the ACPI reduced hardware flag for PVH DomUs, according to the
> description in the ACPI spec:
> 
> "For certain classes of systems the ACPI Hardware Specification may
> not be adequate. Examples include legacy-free, UEFI-based platforms
> with recent processors, and those implementing mobile platform
> architectures. For such platforms, a Hardware-reduced ACPI mode is
> defined."
> 
> Certainly the legacy-free and UEFI part is quite applicable to PVH
> DomU, for which we don't plan to support legacy BIOS and only provide
> UEFI firmware in the long term.
> 
> Reduced HW ACPI also gets rid of the SCI interrupt, and instead
> provides some other methods to signal ACPI events (note we don't
> use any ACPI event ATM for PVH DomU). It also gets rid of a bunch of
> FADT fields that we don't use for PVH DomU either.
> 
> I however seem to remember some past discussion about PVH DomU and
> reduced ACPI, but I cannot find the thread. If there are no objections
> I think we should look into this (likely discuss with the ACPI working
> group) in order to figure out if reduced HW ACPI could work for us,
> and how the event delivery could be implemented for PVH DomU if it
> turns out we need it later on. It might make sense to also figure out
> what other people do, like HyperV Gen2 instances (which also get rid
> of a lot of legacy hw).

Well, without a proper / complete list of implications and restrictions
resulting from that mode I don't think we can take a decision either
way. All I recall is things being scattered all over the spec at the time
I first looked at this a little.

Jan



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: PVH and ACPI discussion

[Roger Pau Monné]

On Fri, Jan 04, 2019 at 06:10:29AM -0700, Jan Beulich wrote:
> >>> On 03.01.19 at 18:45, <roger.pau@citrix.com> wrote:
> > While looking at some tangential issues I realized that the 'VGA Not
> > Present' flag that Xen currently sets for PVH DomUs might be slightly
> > different from what we expect it to mean. The purpose was that Xen
> > would set this flag to denote there's no VGA MMIO region in the low
> > 1MB, so that the guest OS would not reserve memory in that area
> > thinking there's a MMIO window there. The memory map provided to a PVH
> > DomU typically contains a single RAM range that expands from 0 to the
> > selected amount of memory.
> > 
> > The description of such flag by the ACPI spec (6.2A) however is as
> > follows:
> > 
> > "If set, indicates to OSPM that it must not blindly probe the VGA
> > hardware (that responds to MMIO addresses A0000h-BFFFFh and IO ports
> > 3B0h-3BBh and 3C0h-3DFh) that may cause machine check on this system.
> > If clear, indicates to OSPM that it is safe to probe the VGA
> > hardware."
> > 
> > My reading of the above text would make me think that if the flag is
> > set the memory region A0000h-BFFFFh should not be used at all, and
> > that would be in conflict with the memory map that's provided to
> > guests (which lists this area as RAM).
> > 
> > I'm not convinced of the best way to proceed here. I can contact the
> > ACPI working group and try to clarify the meaning of the flag, or
> > inquiry if there's a more suitable flag for or use case, but I would
> > like to hear others opinion on this topic.
> 
> "Should not blindly probe" != "should not use". To me the wording
> implies that some secondary means are necessary to tell what the
> region is used for, which is left unspecified by ACPI itself. If that's
> what's meant, we're free to make our secondary spec "it's RAM in
> all cases".

OK, so in our case this flag together with the memory map provided to
the guest should make it clear this is a RAM region and accesses are
fine.

> > Secondly, I've also been looking at whether it would make sense to set
> > the ACPI reduced hardware flag for PVH DomUs, according to the
> > description in the ACPI spec:
> > 
> > "For certain classes of systems the ACPI Hardware Specification may
> > not be adequate. Examples include legacy-free, UEFI-based platforms
> > with recent processors, and those implementing mobile platform
> > architectures. For such platforms, a Hardware-reduced ACPI mode is
> > defined."
> > 
> > Certainly the legacy-free and UEFI part is quite applicable to PVH
> > DomU, for which we don't plan to support legacy BIOS and only provide
> > UEFI firmware in the long term.
> > 
> > Reduced HW ACPI also gets rid of the SCI interrupt, and instead
> > provides some other methods to signal ACPI events (note we don't
> > use any ACPI event ATM for PVH DomU). It also gets rid of a bunch of
> > FADT fields that we don't use for PVH DomU either.
> > 
> > I however seem to remember some past discussion about PVH DomU and
> > reduced ACPI, but I cannot find the thread. If there are no objections
> > I think we should look into this (likely discuss with the ACPI working
> > group) in order to figure out if reduced HW ACPI could work for us,
> > and how the event delivery could be implemented for PVH DomU if it
> > turns out we need it later on. It might make sense to also figure out
> > what other people do, like HyperV Gen2 instances (which also get rid
> > of a lot of legacy hw).
> 
> Well, without a proper / complete list of implications and restrictions
> resulting from that mode I don't think we can take a decision either
> way. All I recall is things being scattered all over the spec at the time
> I first looked at this a little.

Yes. I've looked at the spec also, and I'm having the same issue. It's
hard to tell exactly what reduced hardware ACPI implies, because
there's no clear list of differences from traditional ACPI, and it's
all scattered around the spec. I will likely send an email to the ACPI
working group about this.

Do you agree however that PVH DomU could maybe use reduced hardware
ACPI while Dom0 using whatever mode (reduced or not) is set by the
native ACPI tables?

Thanks, Roger.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: PVH and ACPI discussion

[Jan Beulich]

>>> On 04.01.19 at 15:09, <roger.pau@citrix.com> wrote:
> Do you agree however that PVH DomU could maybe use reduced hardware
> ACPI while Dom0 using whatever mode (reduced or not) is set by the
> native ACPI tables?

Yes, it's certainly worth exploring.

Jan



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: PVH and ACPI discussion

[Rian Quinn]

[-- Attachment #1.1: Type: text/plain, Size: 825 bytes --]

With the research we are doing using Bareflank, we set both of these flags
and also tell Linux that almost all (but a small part) of the lower 1MB is
free RAM to use as we don't emulate any of the legacy devices. From what we
can tell so far, Linux seems to handle this fine without any complaints.

- Rian

On Fri, Jan 4, 2019 at 8:44 AM Jan Beulich <JBeulich@suse.com> wrote:

> >>> On 04.01.19 at 15:09, <roger.pau@citrix.com> wrote:
> > Do you agree however that PVH DomU could maybe use reduced hardware
> > ACPI while Dom0 using whatever mode (reduced or not) is set by the
> > native ACPI tables?
>
> Yes, it's certainly worth exploring.
>
> Jan
>
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xenproject.org
> https://lists.xenproject.org/mailman/listinfo/xen-devel

[-- Attachment #1.2: Type: text/html, Size: 1414 bytes --]

[-- Attachment #2: Type: text/plain, Size: 157 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel