* Re: [Xen-devel] Xen PVH support in grub2
@ 2017-11-03 14:36 ` Boris Ostrovsky
0 siblings, 0 replies; 73+ messages in thread
From: Boris Ostrovsky @ 2017-11-03 14:36 UTC (permalink / raw)
To: Juergen Gross, Roger Pau Monné
Cc: The development of GNU GRUB, Daniel Kiper, xen-devel
On 11/03/2017 10:24 AM, Juergen Gross wrote:
> On 03/11/17 15:07, Roger Pau Monné wrote:
>> On Fri, Nov 03, 2017 at 01:50:11PM +0100, Juergen Gross wrote:
>>> On 03/11/17 13:17, Roger Pau Monné wrote:
>>>> On Fri, Nov 03, 2017 at 01:00:46PM +0100, Juergen Gross wrote:
>>>>> On 29/09/17 17:51, Roger Pau Monné wrote:
>>>>>> On Fri, Sep 29, 2017 at 03:33:58PM +0000, Juergen Gross wrote:
>>>>>>> On 29/09/17 17:24, Roger Pau Monné wrote:
>>>>>>>> On Fri, Sep 29, 2017 at 02:46:53PM +0000, Juergen Gross wrote:
>>>>>>>> Then, I also wonder whether it would make sense for this grub to load
>>>>>>>> the kernel using the PVH entry point or the native entry point. Would
>>>>>>>> it be possible to boot a Linux kernel up to the point where cpuid can
>>>>>>>> be used inside of a PVH container?
>>>>>>> I don't think today's Linux allows that. This has been discussed
>>>>>>> very thoroughly at the time Boris added PVH V2 support to the kernel.
>>>>>> OK, I'm not going to insist on that, but my plans for FreeBSD is to
>>>>>> make the native entry point capable of booting inside of a PVH
>>>>>> container up to the point where cpuid (or whatever method) can be used
>>>>>> to detect the environment.
>>>>> Looking more thoroughly into the Linux boot code I think this could
>>>>> work for Linux, too. But only if we can tell PVH from HVM in the guest.
>>>>> How would you do that in FreeBSD? Via flags in the boot params? This
>>>>> would the have to be done in the boot loader (e.g. grub or OVMF).
>>>> My plan was not to differentiate between HVM and PVH, but rather to
>>>> make use of the ACPI information in order to decide which devices are
>>>> available and which are not inside of a PVH guest.
>>>>
>>>> For example in the FADT "IA-PC Boot Architecture Flags" field for PVH
>>>> we already set "VGA Not Present" and "CMOS RTC Not Present". There
>>>> might be other flags/fields that must be set, but I would like to
>>>> avoid having a CPUID bit or similar saying "PVH", because then Xen
>>>> will be tied to always providing the same set of devices in PVH
>>>> containers.
>>> Why? This would depend on the semantics tied to the flag. It could just
>>> mean "don't assume availability of legacy stuff" (e.g. BIOS calls).
>>>
>>> Linux would have a problem with the ACPI approach as it would try BIOS
>>> calls way before it is initializing its ACPI handling. So in Linux I'd
>>> need another way to tell I'm running in PVH mode, e.g. a "no legacy"
>>> bit in the Xen HVM cpuid leaf.
>> If you are booted from the PVH entry point, there's no BIOS or UEFI
>> (ie: no firmware), if you are booted from the BIOS entry point there's
>> a BIOS and the same applies to UEFI. How does Linux differentiate
>> whether it's booted from BIOS or UEFI?
> They use different entries.
In fact, we had a discussion with Matt Fleming (Linux EFI maintainer) to
see if we can use EFI entry point to also be able to boot PVH guest but
found some issues with that approach, which is why we ended up with a
dedicated PVH entry point.
I am curious though, Juergen --- what do we need besides zeropage to
allow us to boot PVH from startup_64?
-boris
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: [Xen-devel] Xen PVH support in grub2
2017-11-03 14:36 ` [Xen-devel] " Boris Ostrovsky
(?)
@ 2017-11-03 14:59 ` Juergen Gross
2017-11-03 15:10 ` Boris Ostrovsky
2017-11-03 15:10 ` Boris Ostrovsky
-1 siblings, 2 replies; 73+ messages in thread
From: Juergen Gross @ 2017-11-03 14:59 UTC (permalink / raw)
To: Boris Ostrovsky, Roger Pau Monné
Cc: The development of GNU GRUB, Daniel Kiper, xen-devel
On 03/11/17 15:36, Boris Ostrovsky wrote:
> On 11/03/2017 10:24 AM, Juergen Gross wrote:
>> On 03/11/17 15:07, Roger Pau Monné wrote:
>>> On Fri, Nov 03, 2017 at 01:50:11PM +0100, Juergen Gross wrote:
>>>> On 03/11/17 13:17, Roger Pau Monné wrote:
>>>>> On Fri, Nov 03, 2017 at 01:00:46PM +0100, Juergen Gross wrote:
>>>>>> On 29/09/17 17:51, Roger Pau Monné wrote:
>>>>>>> On Fri, Sep 29, 2017 at 03:33:58PM +0000, Juergen Gross wrote:
>>>>>>>> On 29/09/17 17:24, Roger Pau Monné wrote:
>>>>>>>>> On Fri, Sep 29, 2017 at 02:46:53PM +0000, Juergen Gross wrote:
>>>>>>>>> Then, I also wonder whether it would make sense for this grub to load
>>>>>>>>> the kernel using the PVH entry point or the native entry point. Would
>>>>>>>>> it be possible to boot a Linux kernel up to the point where cpuid can
>>>>>>>>> be used inside of a PVH container?
>>>>>>>> I don't think today's Linux allows that. This has been discussed
>>>>>>>> very thoroughly at the time Boris added PVH V2 support to the kernel.
>>>>>>> OK, I'm not going to insist on that, but my plans for FreeBSD is to
>>>>>>> make the native entry point capable of booting inside of a PVH
>>>>>>> container up to the point where cpuid (or whatever method) can be used
>>>>>>> to detect the environment.
>>>>>> Looking more thoroughly into the Linux boot code I think this could
>>>>>> work for Linux, too. But only if we can tell PVH from HVM in the guest.
>>>>>> How would you do that in FreeBSD? Via flags in the boot params? This
>>>>>> would the have to be done in the boot loader (e.g. grub or OVMF).
>>>>> My plan was not to differentiate between HVM and PVH, but rather to
>>>>> make use of the ACPI information in order to decide which devices are
>>>>> available and which are not inside of a PVH guest.
>>>>>
>>>>> For example in the FADT "IA-PC Boot Architecture Flags" field for PVH
>>>>> we already set "VGA Not Present" and "CMOS RTC Not Present". There
>>>>> might be other flags/fields that must be set, but I would like to
>>>>> avoid having a CPUID bit or similar saying "PVH", because then Xen
>>>>> will be tied to always providing the same set of devices in PVH
>>>>> containers.
>>>> Why? This would depend on the semantics tied to the flag. It could just
>>>> mean "don't assume availability of legacy stuff" (e.g. BIOS calls).
>>>>
>>>> Linux would have a problem with the ACPI approach as it would try BIOS
>>>> calls way before it is initializing its ACPI handling. So in Linux I'd
>>>> need another way to tell I'm running in PVH mode, e.g. a "no legacy"
>>>> bit in the Xen HVM cpuid leaf.
>>> If you are booted from the PVH entry point, there's no BIOS or UEFI
>>> (ie: no firmware), if you are booted from the BIOS entry point there's
>>> a BIOS and the same applies to UEFI. How does Linux differentiate
>>> whether it's booted from BIOS or UEFI?
>> They use different entries.
>
> In fact, we had a discussion with Matt Fleming (Linux EFI maintainer) to
> see if we can use EFI entry point to also be able to boot PVH guest but
> found some issues with that approach, which is why we ended up with a
> dedicated PVH entry point.
>
> I am curious though, Juergen --- what do we need besides zeropage to
> allow us to boot PVH from startup_64?
Oh, you are right. I managed to get lost in the early boot paths.
Only setting up the hyperpage seems to be missing, but this should be
doable. And setting xen_pvh, of course.
Juergen
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: [Xen-devel] Xen PVH support in grub2
2017-11-03 14:59 ` Juergen Gross
@ 2017-11-03 15:10 ` Boris Ostrovsky
2017-11-03 15:27 ` Juergen Gross
2017-11-03 15:27 ` Juergen Gross
2017-11-03 15:10 ` Boris Ostrovsky
1 sibling, 2 replies; 73+ messages in thread
From: Boris Ostrovsky @ 2017-11-03 15:10 UTC (permalink / raw)
To: Juergen Gross, Roger Pau Monné
Cc: The development of GNU GRUB, Daniel Kiper, xen-devel
On 11/03/2017 10:59 AM, Juergen Gross wrote:
> On 03/11/17 15:36, Boris Ostrovsky wrote:
>> On 11/03/2017 10:24 AM, Juergen Gross wrote:
>>> On 03/11/17 15:07, Roger Pau Monné wrote:
>>>> On Fri, Nov 03, 2017 at 01:50:11PM +0100, Juergen Gross wrote:
>>>>> On 03/11/17 13:17, Roger Pau Monné wrote:
>>>>>> On Fri, Nov 03, 2017 at 01:00:46PM +0100, Juergen Gross wrote:
>>>>>>> On 29/09/17 17:51, Roger Pau Monné wrote:
>>>>>>>> On Fri, Sep 29, 2017 at 03:33:58PM +0000, Juergen Gross wrote:
>>>>>>>>> On 29/09/17 17:24, Roger Pau Monné wrote:
>>>>>>>>>> On Fri, Sep 29, 2017 at 02:46:53PM +0000, Juergen Gross wrote:
>>>>>>>>>> Then, I also wonder whether it would make sense for this grub to load
>>>>>>>>>> the kernel using the PVH entry point or the native entry point. Would
>>>>>>>>>> it be possible to boot a Linux kernel up to the point where cpuid can
>>>>>>>>>> be used inside of a PVH container?
>>>>>>>>> I don't think today's Linux allows that. This has been discussed
>>>>>>>>> very thoroughly at the time Boris added PVH V2 support to the kernel.
>>>>>>>> OK, I'm not going to insist on that, but my plans for FreeBSD is to
>>>>>>>> make the native entry point capable of booting inside of a PVH
>>>>>>>> container up to the point where cpuid (or whatever method) can be used
>>>>>>>> to detect the environment.
>>>>>>> Looking more thoroughly into the Linux boot code I think this could
>>>>>>> work for Linux, too. But only if we can tell PVH from HVM in the guest.
>>>>>>> How would you do that in FreeBSD? Via flags in the boot params? This
>>>>>>> would the have to be done in the boot loader (e.g. grub or OVMF).
>>>>>> My plan was not to differentiate between HVM and PVH, but rather to
>>>>>> make use of the ACPI information in order to decide which devices are
>>>>>> available and which are not inside of a PVH guest.
>>>>>>
>>>>>> For example in the FADT "IA-PC Boot Architecture Flags" field for PVH
>>>>>> we already set "VGA Not Present" and "CMOS RTC Not Present". There
>>>>>> might be other flags/fields that must be set, but I would like to
>>>>>> avoid having a CPUID bit or similar saying "PVH", because then Xen
>>>>>> will be tied to always providing the same set of devices in PVH
>>>>>> containers.
>>>>> Why? This would depend on the semantics tied to the flag. It could just
>>>>> mean "don't assume availability of legacy stuff" (e.g. BIOS calls).
>>>>>
>>>>> Linux would have a problem with the ACPI approach as it would try BIOS
>>>>> calls way before it is initializing its ACPI handling. So in Linux I'd
>>>>> need another way to tell I'm running in PVH mode, e.g. a "no legacy"
>>>>> bit in the Xen HVM cpuid leaf.
>>>> If you are booted from the PVH entry point, there's no BIOS or UEFI
>>>> (ie: no firmware), if you are booted from the BIOS entry point there's
>>>> a BIOS and the same applies to UEFI. How does Linux differentiate
>>>> whether it's booted from BIOS or UEFI?
>>> They use different entries.
>> In fact, we had a discussion with Matt Fleming (Linux EFI maintainer) to
>> see if we can use EFI entry point to also be able to boot PVH guest but
>> found some issues with that approach, which is why we ended up with a
>> dedicated PVH entry point.
>>
>> I am curious though, Juergen --- what do we need besides zeropage to
>> allow us to boot PVH from startup_64?
> Oh, you are right. I managed to get lost in the early boot paths.
>
> Only setting up the hyperpage seems to be missing, but this should be
> doable. And setting xen_pvh, of course.
That last part was actually my question --- do we need to have xen_pvh
set before we get to xen-specific code for the first time (which I think
is init_hypervisor_platform()) from startup_64?
Because if we do --- who will set it?
-boris
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: [Xen-devel] Xen PVH support in grub2
2017-11-03 15:10 ` Boris Ostrovsky
@ 2017-11-03 15:27 ` Juergen Gross
2017-11-03 18:05 ` Juergen Gross
2017-11-03 18:05 ` Juergen Gross
2017-11-03 15:27 ` Juergen Gross
1 sibling, 2 replies; 73+ messages in thread
From: Juergen Gross @ 2017-11-03 15:27 UTC (permalink / raw)
To: Boris Ostrovsky, Roger Pau Monné
Cc: The development of GNU GRUB, Daniel Kiper, xen-devel
On 03/11/17 16:10, Boris Ostrovsky wrote:
> On 11/03/2017 10:59 AM, Juergen Gross wrote:
>> On 03/11/17 15:36, Boris Ostrovsky wrote:
>>> On 11/03/2017 10:24 AM, Juergen Gross wrote:
>>>> On 03/11/17 15:07, Roger Pau Monné wrote:
>>>>> On Fri, Nov 03, 2017 at 01:50:11PM +0100, Juergen Gross wrote:
>>>>>> On 03/11/17 13:17, Roger Pau Monné wrote:
>>>>>>> On Fri, Nov 03, 2017 at 01:00:46PM +0100, Juergen Gross wrote:
>>>>>>>> On 29/09/17 17:51, Roger Pau Monné wrote:
>>>>>>>>> On Fri, Sep 29, 2017 at 03:33:58PM +0000, Juergen Gross wrote:
>>>>>>>>>> On 29/09/17 17:24, Roger Pau Monné wrote:
>>>>>>>>>>> On Fri, Sep 29, 2017 at 02:46:53PM +0000, Juergen Gross wrote:
>>>>>>>>>>> Then, I also wonder whether it would make sense for this grub to load
>>>>>>>>>>> the kernel using the PVH entry point or the native entry point. Would
>>>>>>>>>>> it be possible to boot a Linux kernel up to the point where cpuid can
>>>>>>>>>>> be used inside of a PVH container?
>>>>>>>>>> I don't think today's Linux allows that. This has been discussed
>>>>>>>>>> very thoroughly at the time Boris added PVH V2 support to the kernel.
>>>>>>>>> OK, I'm not going to insist on that, but my plans for FreeBSD is to
>>>>>>>>> make the native entry point capable of booting inside of a PVH
>>>>>>>>> container up to the point where cpuid (or whatever method) can be used
>>>>>>>>> to detect the environment.
>>>>>>>> Looking more thoroughly into the Linux boot code I think this could
>>>>>>>> work for Linux, too. But only if we can tell PVH from HVM in the guest.
>>>>>>>> How would you do that in FreeBSD? Via flags in the boot params? This
>>>>>>>> would the have to be done in the boot loader (e.g. grub or OVMF).
>>>>>>> My plan was not to differentiate between HVM and PVH, but rather to
>>>>>>> make use of the ACPI information in order to decide which devices are
>>>>>>> available and which are not inside of a PVH guest.
>>>>>>>
>>>>>>> For example in the FADT "IA-PC Boot Architecture Flags" field for PVH
>>>>>>> we already set "VGA Not Present" and "CMOS RTC Not Present". There
>>>>>>> might be other flags/fields that must be set, but I would like to
>>>>>>> avoid having a CPUID bit or similar saying "PVH", because then Xen
>>>>>>> will be tied to always providing the same set of devices in PVH
>>>>>>> containers.
>>>>>> Why? This would depend on the semantics tied to the flag. It could just
>>>>>> mean "don't assume availability of legacy stuff" (e.g. BIOS calls).
>>>>>>
>>>>>> Linux would have a problem with the ACPI approach as it would try BIOS
>>>>>> calls way before it is initializing its ACPI handling. So in Linux I'd
>>>>>> need another way to tell I'm running in PVH mode, e.g. a "no legacy"
>>>>>> bit in the Xen HVM cpuid leaf.
>>>>> If you are booted from the PVH entry point, there's no BIOS or UEFI
>>>>> (ie: no firmware), if you are booted from the BIOS entry point there's
>>>>> a BIOS and the same applies to UEFI. How does Linux differentiate
>>>>> whether it's booted from BIOS or UEFI?
>>>> They use different entries.
>>> In fact, we had a discussion with Matt Fleming (Linux EFI maintainer) to
>>> see if we can use EFI entry point to also be able to boot PVH guest but
>>> found some issues with that approach, which is why we ended up with a
>>> dedicated PVH entry point.
>>>
>>> I am curious though, Juergen --- what do we need besides zeropage to
>>> allow us to boot PVH from startup_64?
>> Oh, you are right. I managed to get lost in the early boot paths.
>>
>> Only setting up the hyperpage seems to be missing, but this should be
>> doable. And setting xen_pvh, of course.
>
> That last part was actually my question --- do we need to have xen_pvh
> set before we get to xen-specific code for the first time (which I think
> is init_hypervisor_platform()) from startup_64?
>
> Because if we do --- who will set it?
This should be easily testable: Just set an internal indicator in
xen_prepare_pvh() and copy that to xen_pvh in init_hypervisor_platform()
and see whether the kernel still comes up in PVH mode.
The same can be done with setting up the hypercall page.
Just building a kernel trying that...
Juergen
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: [Xen-devel] Xen PVH support in grub2
2017-11-03 15:27 ` Juergen Gross
@ 2017-11-03 18:05 ` Juergen Gross
2017-11-03 18:19 ` Boris Ostrovsky
2017-11-03 18:19 ` [Xen-devel] " Boris Ostrovsky
2017-11-03 18:05 ` Juergen Gross
1 sibling, 2 replies; 73+ messages in thread
From: Juergen Gross @ 2017-11-03 18:05 UTC (permalink / raw)
To: Boris Ostrovsky, Roger Pau Monné
Cc: The development of GNU GRUB, Daniel Kiper, xen-devel
On 03/11/17 16:27, Juergen Gross wrote:
> On 03/11/17 16:10, Boris Ostrovsky wrote:
>> On 11/03/2017 10:59 AM, Juergen Gross wrote:
>>> On 03/11/17 15:36, Boris Ostrovsky wrote:
>>>> On 11/03/2017 10:24 AM, Juergen Gross wrote:
>>>>> On 03/11/17 15:07, Roger Pau Monné wrote:
>>>>>> On Fri, Nov 03, 2017 at 01:50:11PM +0100, Juergen Gross wrote:
>>>>>>> On 03/11/17 13:17, Roger Pau Monné wrote:
>>>>>>>> On Fri, Nov 03, 2017 at 01:00:46PM +0100, Juergen Gross wrote:
>>>>>>>>> On 29/09/17 17:51, Roger Pau Monné wrote:
>>>>>>>>>> On Fri, Sep 29, 2017 at 03:33:58PM +0000, Juergen Gross wrote:
>>>>>>>>>>> On 29/09/17 17:24, Roger Pau Monné wrote:
>>>>>>>>>>>> On Fri, Sep 29, 2017 at 02:46:53PM +0000, Juergen Gross wrote:
>>>>>>>>>>>> Then, I also wonder whether it would make sense for this grub to load
>>>>>>>>>>>> the kernel using the PVH entry point or the native entry point. Would
>>>>>>>>>>>> it be possible to boot a Linux kernel up to the point where cpuid can
>>>>>>>>>>>> be used inside of a PVH container?
>>>>>>>>>>> I don't think today's Linux allows that. This has been discussed
>>>>>>>>>>> very thoroughly at the time Boris added PVH V2 support to the kernel.
>>>>>>>>>> OK, I'm not going to insist on that, but my plans for FreeBSD is to
>>>>>>>>>> make the native entry point capable of booting inside of a PVH
>>>>>>>>>> container up to the point where cpuid (or whatever method) can be used
>>>>>>>>>> to detect the environment.
>>>>>>>>> Looking more thoroughly into the Linux boot code I think this could
>>>>>>>>> work for Linux, too. But only if we can tell PVH from HVM in the guest.
>>>>>>>>> How would you do that in FreeBSD? Via flags in the boot params? This
>>>>>>>>> would the have to be done in the boot loader (e.g. grub or OVMF).
>>>>>>>> My plan was not to differentiate between HVM and PVH, but rather to
>>>>>>>> make use of the ACPI information in order to decide which devices are
>>>>>>>> available and which are not inside of a PVH guest.
>>>>>>>>
>>>>>>>> For example in the FADT "IA-PC Boot Architecture Flags" field for PVH
>>>>>>>> we already set "VGA Not Present" and "CMOS RTC Not Present". There
>>>>>>>> might be other flags/fields that must be set, but I would like to
>>>>>>>> avoid having a CPUID bit or similar saying "PVH", because then Xen
>>>>>>>> will be tied to always providing the same set of devices in PVH
>>>>>>>> containers.
>>>>>>> Why? This would depend on the semantics tied to the flag. It could just
>>>>>>> mean "don't assume availability of legacy stuff" (e.g. BIOS calls).
>>>>>>>
>>>>>>> Linux would have a problem with the ACPI approach as it would try BIOS
>>>>>>> calls way before it is initializing its ACPI handling. So in Linux I'd
>>>>>>> need another way to tell I'm running in PVH mode, e.g. a "no legacy"
>>>>>>> bit in the Xen HVM cpuid leaf.
>>>>>> If you are booted from the PVH entry point, there's no BIOS or UEFI
>>>>>> (ie: no firmware), if you are booted from the BIOS entry point there's
>>>>>> a BIOS and the same applies to UEFI. How does Linux differentiate
>>>>>> whether it's booted from BIOS or UEFI?
>>>>> They use different entries.
>>>> In fact, we had a discussion with Matt Fleming (Linux EFI maintainer) to
>>>> see if we can use EFI entry point to also be able to boot PVH guest but
>>>> found some issues with that approach, which is why we ended up with a
>>>> dedicated PVH entry point.
>>>>
>>>> I am curious though, Juergen --- what do we need besides zeropage to
>>>> allow us to boot PVH from startup_64?
>>> Oh, you are right. I managed to get lost in the early boot paths.
>>>
>>> Only setting up the hyperpage seems to be missing, but this should be
>>> doable. And setting xen_pvh, of course.
>>
>> That last part was actually my question --- do we need to have xen_pvh
>> set before we get to xen-specific code for the first time (which I think
>> is init_hypervisor_platform()) from startup_64?
>>
>> Because if we do --- who will set it?
>
> This should be easily testable: Just set an internal indicator in
> xen_prepare_pvh() and copy that to xen_pvh in init_hypervisor_platform()
> and see whether the kernel still comes up in PVH mode.
>
> The same can be done with setting up the hypercall page.
>
> Just building a kernel trying that...
Okay, setting xen_pvh late is working. I can't omit setting up the
hypercall_page right now, of course, as then I wouldn't get the memory
map. But this shouldn't be a problem for the grub/OVMF case as those
would setup the memory map for the guest in zeropage.
So again the question: how to tell whether we are PVH or HVM in
init_hypervisor_platform()? ACPi tables are scanned way later...
Juergen
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: Xen PVH support in grub2
2017-11-03 18:05 ` Juergen Gross
@ 2017-11-03 18:19 ` Boris Ostrovsky
2017-11-03 18:19 ` [Xen-devel] " Boris Ostrovsky
1 sibling, 0 replies; 73+ messages in thread
From: Boris Ostrovsky @ 2017-11-03 18:19 UTC (permalink / raw)
To: Juergen Gross, Roger Pau Monné
Cc: The development of GNU GRUB, Daniel Kiper, xen-devel
On 11/03/2017 02:05 PM, Juergen Gross wrote:
>
> So again the question: how to tell whether we are PVH or HVM in
> init_hypervisor_platform()? ACPi tables are scanned way later...
Can we make grub/OVMF append a boot option?
Or set setup_header.hardware_subarch to something? We already have
X86_SUBARCH_XEN but it is only used by PV. Or we might be able to use
hardware_subarch_data (will need to get a buy-in from x86 maintainers, I
think).
-boris
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: [Xen-devel] Xen PVH support in grub2
2017-11-03 18:05 ` Juergen Gross
2017-11-03 18:19 ` Boris Ostrovsky
@ 2017-11-03 18:19 ` Boris Ostrovsky
2017-11-03 18:23 ` Juergen Gross
2017-11-03 18:23 ` Juergen Gross
1 sibling, 2 replies; 73+ messages in thread
From: Boris Ostrovsky @ 2017-11-03 18:19 UTC (permalink / raw)
To: Juergen Gross, Roger Pau Monné
Cc: The development of GNU GRUB, Daniel Kiper, xen-devel
On 11/03/2017 02:05 PM, Juergen Gross wrote:
>
> So again the question: how to tell whether we are PVH or HVM in
> init_hypervisor_platform()? ACPi tables are scanned way later...
Can we make grub/OVMF append a boot option?
Or set setup_header.hardware_subarch to something? We already have
X86_SUBARCH_XEN but it is only used by PV. Or we might be able to use
hardware_subarch_data (will need to get a buy-in from x86 maintainers, I
think).
-boris
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: [Xen-devel] Xen PVH support in grub2
2017-11-03 18:19 ` [Xen-devel] " Boris Ostrovsky
@ 2017-11-03 18:23 ` Juergen Gross
2017-11-03 18:35 ` Boris Ostrovsky
` (2 more replies)
2017-11-03 18:23 ` Juergen Gross
1 sibling, 3 replies; 73+ messages in thread
From: Juergen Gross @ 2017-11-03 18:23 UTC (permalink / raw)
To: Boris Ostrovsky, Roger Pau Monné
Cc: The development of GNU GRUB, Daniel Kiper, xen-devel
On 03/11/17 19:19, Boris Ostrovsky wrote:
> On 11/03/2017 02:05 PM, Juergen Gross wrote:
>>
>> So again the question: how to tell whether we are PVH or HVM in
>> init_hypervisor_platform()? ACPi tables are scanned way later...
>
> Can we make grub/OVMF append a boot option?
>
> Or set setup_header.hardware_subarch to something? We already have
> X86_SUBARCH_XEN but it is only used by PV. Or we might be able to use
> hardware_subarch_data (will need to get a buy-in from x86 maintainers, I
> think).
But wouldn't this break the idea to reuse the native boot paths in
grub/OVMF without further modifications?
I'd rather have a way to ask the hypervisor whether we are in PVH mode
(e.g. via CPUID or a hypercall to test for a devicemodel having itself
registered).
In case we need to set an indicator in zeropage I'd opt for reusing
X86_SUBARCH_XEN.
Juergen
^ permalink raw reply [flat|nested] 73+ messages in thread* Re: [Xen-devel] Xen PVH support in grub2
2017-11-03 18:23 ` Juergen Gross
@ 2017-11-03 18:35 ` Boris Ostrovsky
2017-11-03 18:40 ` Juergen Gross
2017-11-03 18:40 ` Juergen Gross
2017-11-03 18:35 ` Boris Ostrovsky
2017-11-03 18:37 ` [Xen-devel] " Roger Pau Monné
2 siblings, 2 replies; 73+ messages in thread
From: Boris Ostrovsky @ 2017-11-03 18:35 UTC (permalink / raw)
To: Juergen Gross, Roger Pau Monné
Cc: The development of GNU GRUB, Daniel Kiper, xen-devel
On 11/03/2017 02:23 PM, Juergen Gross wrote:
> On 03/11/17 19:19, Boris Ostrovsky wrote:
>> On 11/03/2017 02:05 PM, Juergen Gross wrote:
>>> So again the question: how to tell whether we are PVH or HVM in
>>> init_hypervisor_platform()? ACPi tables are scanned way later...
>> Can we make grub/OVMF append a boot option?
>>
>> Or set setup_header.hardware_subarch to something? We already have
>> X86_SUBARCH_XEN but it is only used by PV. Or we might be able to use
>> hardware_subarch_data (will need to get a buy-in from x86 maintainers, I
>> think).
> But wouldn't this break the idea to reuse the native boot paths in
> grub/OVMF without further modifications?
WDYM? We will have to have some sort of a plugin in either one to build
the zeropage anyway. So we'd set hardware_subarch there, in addition to
other things like setting memory and such.
-boris
>
> I'd rather have a way to ask the hypervisor whether we are in PVH mode
> (e.g. via CPUID or a hypercall to test for a devicemodel having itself
> registered).
>
> In case we need to set an indicator in zeropage I'd opt for reusing
> X86_SUBARCH_XEN.
>
>
> Juergen
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: [Xen-devel] Xen PVH support in grub2
2017-11-03 18:35 ` Boris Ostrovsky
@ 2017-11-03 18:40 ` Juergen Gross
2017-11-03 19:00 ` Boris Ostrovsky
2017-11-03 19:00 ` [Xen-devel] " Boris Ostrovsky
2017-11-03 18:40 ` Juergen Gross
1 sibling, 2 replies; 73+ messages in thread
From: Juergen Gross @ 2017-11-03 18:40 UTC (permalink / raw)
To: Boris Ostrovsky, Roger Pau Monné
Cc: The development of GNU GRUB, Daniel Kiper, xen-devel
On 03/11/17 19:35, Boris Ostrovsky wrote:
> On 11/03/2017 02:23 PM, Juergen Gross wrote:
>> On 03/11/17 19:19, Boris Ostrovsky wrote:
>>> On 11/03/2017 02:05 PM, Juergen Gross wrote:
>>>> So again the question: how to tell whether we are PVH or HVM in
>>>> init_hypervisor_platform()? ACPi tables are scanned way later...
>>> Can we make grub/OVMF append a boot option?
>>>
>>> Or set setup_header.hardware_subarch to something? We already have
>>> X86_SUBARCH_XEN but it is only used by PV. Or we might be able to use
>>> hardware_subarch_data (will need to get a buy-in from x86 maintainers, I
>>> think).
>> But wouldn't this break the idea to reuse the native boot paths in
>> grub/OVMF without further modifications?
>
> WDYM? We will have to have some sort of a plugin in either one to build
> the zeropage anyway. So we'd set hardware_subarch there, in addition to
> other things like setting memory and such.
But isn't the zeropage already being built? I admit that setting subarch
isn't a big deal, but using another entry with a passed-through pvh
start struct isn't either...
Juergen
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: Xen PVH support in grub2
2017-11-03 18:40 ` Juergen Gross
@ 2017-11-03 19:00 ` Boris Ostrovsky
2017-11-03 19:00 ` [Xen-devel] " Boris Ostrovsky
1 sibling, 0 replies; 73+ messages in thread
From: Boris Ostrovsky @ 2017-11-03 19:00 UTC (permalink / raw)
To: Juergen Gross, Roger Pau Monné
Cc: The development of GNU GRUB, Daniel Kiper, xen-devel
On 11/03/2017 02:40 PM, Juergen Gross wrote:
> On 03/11/17 19:35, Boris Ostrovsky wrote:
>> On 11/03/2017 02:23 PM, Juergen Gross wrote:
>>> On 03/11/17 19:19, Boris Ostrovsky wrote:
>>>> On 11/03/2017 02:05 PM, Juergen Gross wrote:
>>>>> So again the question: how to tell whether we are PVH or HVM in
>>>>> init_hypervisor_platform()? ACPi tables are scanned way later...
>>>> Can we make grub/OVMF append a boot option?
>>>>
>>>> Or set setup_header.hardware_subarch to something? We already have
>>>> X86_SUBARCH_XEN but it is only used by PV. Or we might be able to use
>>>> hardware_subarch_data (will need to get a buy-in from x86 maintainers, I
>>>> think).
>>> But wouldn't this break the idea to reuse the native boot paths in
>>> grub/OVMF without further modifications?
>> WDYM? We will have to have some sort of a plugin in either one to build
>> the zeropage anyway. So we'd set hardware_subarch there, in addition to
>> other things like setting memory and such.
> But isn't the zeropage already being built? I admit that setting subarch
> isn't a big deal, but using another entry with a passed-through pvh
> start struct isn't either...
I don't follow, sorry. My understanding is that zeropage will be built
by PVH-enlightened grub so part of this process would be setting the
subarch bit.
-boris
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: [Xen-devel] Xen PVH support in grub2
2017-11-03 18:40 ` Juergen Gross
2017-11-03 19:00 ` Boris Ostrovsky
@ 2017-11-03 19:00 ` Boris Ostrovsky
2017-11-06 7:16 ` Juergen Gross
2017-11-06 7:16 ` [Xen-devel] " Juergen Gross
1 sibling, 2 replies; 73+ messages in thread
From: Boris Ostrovsky @ 2017-11-03 19:00 UTC (permalink / raw)
To: Juergen Gross, Roger Pau Monné
Cc: The development of GNU GRUB, Daniel Kiper, xen-devel
On 11/03/2017 02:40 PM, Juergen Gross wrote:
> On 03/11/17 19:35, Boris Ostrovsky wrote:
>> On 11/03/2017 02:23 PM, Juergen Gross wrote:
>>> On 03/11/17 19:19, Boris Ostrovsky wrote:
>>>> On 11/03/2017 02:05 PM, Juergen Gross wrote:
>>>>> So again the question: how to tell whether we are PVH or HVM in
>>>>> init_hypervisor_platform()? ACPi tables are scanned way later...
>>>> Can we make grub/OVMF append a boot option?
>>>>
>>>> Or set setup_header.hardware_subarch to something? We already have
>>>> X86_SUBARCH_XEN but it is only used by PV. Or we might be able to use
>>>> hardware_subarch_data (will need to get a buy-in from x86 maintainers, I
>>>> think).
>>> But wouldn't this break the idea to reuse the native boot paths in
>>> grub/OVMF without further modifications?
>> WDYM? We will have to have some sort of a plugin in either one to build
>> the zeropage anyway. So we'd set hardware_subarch there, in addition to
>> other things like setting memory and such.
> But isn't the zeropage already being built? I admit that setting subarch
> isn't a big deal, but using another entry with a passed-through pvh
> start struct isn't either...
I don't follow, sorry. My understanding is that zeropage will be built
by PVH-enlightened grub so part of this process would be setting the
subarch bit.
-boris
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: Xen PVH support in grub2
2017-11-03 19:00 ` [Xen-devel] " Boris Ostrovsky
@ 2017-11-06 7:16 ` Juergen Gross
2017-11-06 7:16 ` [Xen-devel] " Juergen Gross
1 sibling, 0 replies; 73+ messages in thread
From: Juergen Gross @ 2017-11-06 7:16 UTC (permalink / raw)
To: Boris Ostrovsky, Roger Pau Monné
Cc: The development of GNU GRUB, Daniel Kiper, xen-devel
On 03/11/17 20:00, Boris Ostrovsky wrote:
> On 11/03/2017 02:40 PM, Juergen Gross wrote:
>> On 03/11/17 19:35, Boris Ostrovsky wrote:
>>> On 11/03/2017 02:23 PM, Juergen Gross wrote:
>>>> On 03/11/17 19:19, Boris Ostrovsky wrote:
>>>>> On 11/03/2017 02:05 PM, Juergen Gross wrote:
>>>>>> So again the question: how to tell whether we are PVH or HVM in
>>>>>> init_hypervisor_platform()? ACPi tables are scanned way later...
>>>>> Can we make grub/OVMF append a boot option?
>>>>>
>>>>> Or set setup_header.hardware_subarch to something? We already have
>>>>> X86_SUBARCH_XEN but it is only used by PV. Or we might be able to use
>>>>> hardware_subarch_data (will need to get a buy-in from x86 maintainers, I
>>>>> think).
>>>> But wouldn't this break the idea to reuse the native boot paths in
>>>> grub/OVMF without further modifications?
>>> WDYM? We will have to have some sort of a plugin in either one to build
>>> the zeropage anyway. So we'd set hardware_subarch there, in addition to
>>> other things like setting memory and such.
>> But isn't the zeropage already being built? I admit that setting subarch
>> isn't a big deal, but using another entry with a passed-through pvh
>> start struct isn't either...
>
> I don't follow, sorry. My understanding is that zeropage will be built
> by PVH-enlightened grub so part of this process would be setting the
> subarch bit.
My reasoning was based on Roger's remark:
"OTOH if Linux is capable of booting from the native entry point inside
of a PVH container, we would only have to port OVMF and grub in order
to work inside of a PVH container, leaving the rest of the logic
untouched."
Juergen
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: [Xen-devel] Xen PVH support in grub2
2017-11-03 19:00 ` [Xen-devel] " Boris Ostrovsky
2017-11-06 7:16 ` Juergen Gross
@ 2017-11-06 7:16 ` Juergen Gross
2017-11-06 14:51 ` Boris Ostrovsky
2017-11-06 14:51 ` [Xen-devel] " Boris Ostrovsky
1 sibling, 2 replies; 73+ messages in thread
From: Juergen Gross @ 2017-11-06 7:16 UTC (permalink / raw)
To: Boris Ostrovsky, Roger Pau Monné
Cc: The development of GNU GRUB, Daniel Kiper, xen-devel
On 03/11/17 20:00, Boris Ostrovsky wrote:
> On 11/03/2017 02:40 PM, Juergen Gross wrote:
>> On 03/11/17 19:35, Boris Ostrovsky wrote:
>>> On 11/03/2017 02:23 PM, Juergen Gross wrote:
>>>> On 03/11/17 19:19, Boris Ostrovsky wrote:
>>>>> On 11/03/2017 02:05 PM, Juergen Gross wrote:
>>>>>> So again the question: how to tell whether we are PVH or HVM in
>>>>>> init_hypervisor_platform()? ACPi tables are scanned way later...
>>>>> Can we make grub/OVMF append a boot option?
>>>>>
>>>>> Or set setup_header.hardware_subarch to something? We already have
>>>>> X86_SUBARCH_XEN but it is only used by PV. Or we might be able to use
>>>>> hardware_subarch_data (will need to get a buy-in from x86 maintainers, I
>>>>> think).
>>>> But wouldn't this break the idea to reuse the native boot paths in
>>>> grub/OVMF without further modifications?
>>> WDYM? We will have to have some sort of a plugin in either one to build
>>> the zeropage anyway. So we'd set hardware_subarch there, in addition to
>>> other things like setting memory and such.
>> But isn't the zeropage already being built? I admit that setting subarch
>> isn't a big deal, but using another entry with a passed-through pvh
>> start struct isn't either...
>
> I don't follow, sorry. My understanding is that zeropage will be built
> by PVH-enlightened grub so part of this process would be setting the
> subarch bit.
My reasoning was based on Roger's remark:
"OTOH if Linux is capable of booting from the native entry point inside
of a PVH container, we would only have to port OVMF and grub in order
to work inside of a PVH container, leaving the rest of the logic
untouched."
Juergen
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: Xen PVH support in grub2
2017-11-06 7:16 ` [Xen-devel] " Juergen Gross
@ 2017-11-06 14:51 ` Boris Ostrovsky
2017-11-06 14:51 ` [Xen-devel] " Boris Ostrovsky
1 sibling, 0 replies; 73+ messages in thread
From: Boris Ostrovsky @ 2017-11-06 14:51 UTC (permalink / raw)
To: Juergen Gross, Roger Pau Monné
Cc: The development of GNU GRUB, Daniel Kiper, xen-devel
On 11/06/2017 02:16 AM, Juergen Gross wrote:
> On 03/11/17 20:00, Boris Ostrovsky wrote:
>> On 11/03/2017 02:40 PM, Juergen Gross wrote:
>>> On 03/11/17 19:35, Boris Ostrovsky wrote:
>>>> On 11/03/2017 02:23 PM, Juergen Gross wrote:
>>>>> On 03/11/17 19:19, Boris Ostrovsky wrote:
>>>>>> On 11/03/2017 02:05 PM, Juergen Gross wrote:
>>>>>>> So again the question: how to tell whether we are PVH or HVM in
>>>>>>> init_hypervisor_platform()? ACPi tables are scanned way later...
>>>>>> Can we make grub/OVMF append a boot option?
>>>>>>
>>>>>> Or set setup_header.hardware_subarch to something? We already have
>>>>>> X86_SUBARCH_XEN but it is only used by PV. Or we might be able to use
>>>>>> hardware_subarch_data (will need to get a buy-in from x86 maintainers, I
>>>>>> think).
>>>>> But wouldn't this break the idea to reuse the native boot paths in
>>>>> grub/OVMF without further modifications?
>>>> WDYM? We will have to have some sort of a plugin in either one to build
>>>> the zeropage anyway. So we'd set hardware_subarch there, in addition to
>>>> other things like setting memory and such.
>>> But isn't the zeropage already being built? I admit that setting subarch
>>> isn't a big deal, but using another entry with a passed-through pvh
>>> start struct isn't either...
>> I don't follow, sorry. My understanding is that zeropage will be built
>> by PVH-enlightened grub so part of this process would be setting the
>> subarch bit.
> My reasoning was based on Roger's remark:
>
> "OTOH if Linux is capable of booting from the native entry point inside
> of a PVH container, we would only have to port OVMF and grub in order
> to work inside of a PVH container, leaving the rest of the logic
> untouched."
Right, and in my mind porting OVMF/grub includes creating proper zeropage.
BTW, another option might be to "type_of_loader = (9 << 4) | 0", which
is what init_pvh_bootparams() does. In fact, whatever is done in the
firmware should probably match what that routine does.
-boris
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: [Xen-devel] Xen PVH support in grub2
2017-11-06 7:16 ` [Xen-devel] " Juergen Gross
2017-11-06 14:51 ` Boris Ostrovsky
@ 2017-11-06 14:51 ` Boris Ostrovsky
2017-11-06 15:05 ` Juergen Gross
2017-11-06 15:05 ` Juergen Gross
1 sibling, 2 replies; 73+ messages in thread
From: Boris Ostrovsky @ 2017-11-06 14:51 UTC (permalink / raw)
To: Juergen Gross, Roger Pau Monné
Cc: The development of GNU GRUB, Daniel Kiper, xen-devel
On 11/06/2017 02:16 AM, Juergen Gross wrote:
> On 03/11/17 20:00, Boris Ostrovsky wrote:
>> On 11/03/2017 02:40 PM, Juergen Gross wrote:
>>> On 03/11/17 19:35, Boris Ostrovsky wrote:
>>>> On 11/03/2017 02:23 PM, Juergen Gross wrote:
>>>>> On 03/11/17 19:19, Boris Ostrovsky wrote:
>>>>>> On 11/03/2017 02:05 PM, Juergen Gross wrote:
>>>>>>> So again the question: how to tell whether we are PVH or HVM in
>>>>>>> init_hypervisor_platform()? ACPi tables are scanned way later...
>>>>>> Can we make grub/OVMF append a boot option?
>>>>>>
>>>>>> Or set setup_header.hardware_subarch to something? We already have
>>>>>> X86_SUBARCH_XEN but it is only used by PV. Or we might be able to use
>>>>>> hardware_subarch_data (will need to get a buy-in from x86 maintainers, I
>>>>>> think).
>>>>> But wouldn't this break the idea to reuse the native boot paths in
>>>>> grub/OVMF without further modifications?
>>>> WDYM? We will have to have some sort of a plugin in either one to build
>>>> the zeropage anyway. So we'd set hardware_subarch there, in addition to
>>>> other things like setting memory and such.
>>> But isn't the zeropage already being built? I admit that setting subarch
>>> isn't a big deal, but using another entry with a passed-through pvh
>>> start struct isn't either...
>> I don't follow, sorry. My understanding is that zeropage will be built
>> by PVH-enlightened grub so part of this process would be setting the
>> subarch bit.
> My reasoning was based on Roger's remark:
>
> "OTOH if Linux is capable of booting from the native entry point inside
> of a PVH container, we would only have to port OVMF and grub in order
> to work inside of a PVH container, leaving the rest of the logic
> untouched."
Right, and in my mind porting OVMF/grub includes creating proper zeropage.
BTW, another option might be to "type_of_loader = (9 << 4) | 0", which
is what init_pvh_bootparams() does. In fact, whatever is done in the
firmware should probably match what that routine does.
-boris
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: [Xen-devel] Xen PVH support in grub2
2017-11-06 14:51 ` [Xen-devel] " Boris Ostrovsky
@ 2017-11-06 15:05 ` Juergen Gross
2017-11-06 16:42 ` Boris Ostrovsky
2017-11-06 16:42 ` Boris Ostrovsky
2017-11-06 15:05 ` Juergen Gross
1 sibling, 2 replies; 73+ messages in thread
From: Juergen Gross @ 2017-11-06 15:05 UTC (permalink / raw)
To: Boris Ostrovsky, Roger Pau Monné
Cc: The development of GNU GRUB, Daniel Kiper, xen-devel
On 06/11/17 15:51, Boris Ostrovsky wrote:
> On 11/06/2017 02:16 AM, Juergen Gross wrote:
>> On 03/11/17 20:00, Boris Ostrovsky wrote:
>>> On 11/03/2017 02:40 PM, Juergen Gross wrote:
>>>> On 03/11/17 19:35, Boris Ostrovsky wrote:
>>>>> On 11/03/2017 02:23 PM, Juergen Gross wrote:
>>>>>> On 03/11/17 19:19, Boris Ostrovsky wrote:
>>>>>>> On 11/03/2017 02:05 PM, Juergen Gross wrote:
>>>>>>>> So again the question: how to tell whether we are PVH or HVM in
>>>>>>>> init_hypervisor_platform()? ACPi tables are scanned way later...
>>>>>>> Can we make grub/OVMF append a boot option?
>>>>>>>
>>>>>>> Or set setup_header.hardware_subarch to something? We already have
>>>>>>> X86_SUBARCH_XEN but it is only used by PV. Or we might be able to use
>>>>>>> hardware_subarch_data (will need to get a buy-in from x86 maintainers, I
>>>>>>> think).
>>>>>> But wouldn't this break the idea to reuse the native boot paths in
>>>>>> grub/OVMF without further modifications?
>>>>> WDYM? We will have to have some sort of a plugin in either one to build
>>>>> the zeropage anyway. So we'd set hardware_subarch there, in addition to
>>>>> other things like setting memory and such.
>>>> But isn't the zeropage already being built? I admit that setting subarch
>>>> isn't a big deal, but using another entry with a passed-through pvh
>>>> start struct isn't either...
>>> I don't follow, sorry. My understanding is that zeropage will be built
>>> by PVH-enlightened grub so part of this process would be setting the
>>> subarch bit.
>> My reasoning was based on Roger's remark:
>>
>> "OTOH if Linux is capable of booting from the native entry point inside
>> of a PVH container, we would only have to port OVMF and grub in order
>> to work inside of a PVH container, leaving the rest of the logic
>> untouched."
>
> Right, and in my mind porting OVMF/grub includes creating proper zeropage.
Aah, okay. I reasoned on the assumption to just enable OVMF/grub to run
in PVH environment without touching the parts setting up anything for
the new kernel.
> BTW, another option might be to "type_of_loader = (9 << 4) | 0", which
> is what init_pvh_bootparams() does. In fact, whatever is done in the
> firmware should probably match what that routine does.
So it wouldn't be possible any longer to tell whether the kernel has
been booted directly or via grub. I don't like this. The loader type
is accessible via sysfs after all.
Juergen
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: [Xen-devel] Xen PVH support in grub2
2017-11-06 15:05 ` Juergen Gross
@ 2017-11-06 16:42 ` Boris Ostrovsky
2017-11-07 7:42 ` Juergen Gross
2017-11-07 7:42 ` [Xen-devel] " Juergen Gross
2017-11-06 16:42 ` Boris Ostrovsky
1 sibling, 2 replies; 73+ messages in thread
From: Boris Ostrovsky @ 2017-11-06 16:42 UTC (permalink / raw)
To: Juergen Gross, Roger Pau Monné
Cc: The development of GNU GRUB, Daniel Kiper, xen-devel,
Konrad Rzeszutek Wilk
On 11/06/2017 10:05 AM, Juergen Gross wrote:
> On 06/11/17 15:51, Boris Ostrovsky wrote:
>> On 11/06/2017 02:16 AM, Juergen Gross wrote:
>>> On 03/11/17 20:00, Boris Ostrovsky wrote:
>>>> On 11/03/2017 02:40 PM, Juergen Gross wrote:
>>>>> On 03/11/17 19:35, Boris Ostrovsky wrote:
>>>>>> On 11/03/2017 02:23 PM, Juergen Gross wrote:
>>>>>>> On 03/11/17 19:19, Boris Ostrovsky wrote:
>>>>>>>> On 11/03/2017 02:05 PM, Juergen Gross wrote:
>>>>>>>>> So again the question: how to tell whether we are PVH or HVM in
>>>>>>>>> init_hypervisor_platform()? ACPi tables are scanned way later...
>>>>>>>> Can we make grub/OVMF append a boot option?
>>>>>>>>
>>>>>>>> Or set setup_header.hardware_subarch to something? We already have
>>>>>>>> X86_SUBARCH_XEN but it is only used by PV. Or we might be able to use
>>>>>>>> hardware_subarch_data (will need to get a buy-in from x86 maintainers, I
>>>>>>>> think).
>>>>>>> But wouldn't this break the idea to reuse the native boot paths in
>>>>>>> grub/OVMF without further modifications?
>>>>>> WDYM? We will have to have some sort of a plugin in either one to build
>>>>>> the zeropage anyway. So we'd set hardware_subarch there, in addition to
>>>>>> other things like setting memory and such.
>>>>> But isn't the zeropage already being built? I admit that setting subarch
>>>>> isn't a big deal, but using another entry with a passed-through pvh
>>>>> start struct isn't either...
>>>> I don't follow, sorry. My understanding is that zeropage will be built
>>>> by PVH-enlightened grub so part of this process would be setting the
>>>> subarch bit.
>>> My reasoning was based on Roger's remark:
>>>
>>> "OTOH if Linux is capable of booting from the native entry point inside
>>> of a PVH container, we would only have to port OVMF and grub in order
>>> to work inside of a PVH container, leaving the rest of the logic
>>> untouched."
>> Right, and in my mind porting OVMF/grub includes creating proper zeropage.
> Aah, okay. I reasoned on the assumption to just enable OVMF/grub to run
> in PVH environment without touching the parts setting up anything for
> the new kernel.
Someone needs to do what xen_prepare_pvh() does.
And, for 64-bit, we also may need to build early pagetables since
startup_64() (unlike startup_32()) expects paging to be on. (I don't
know whether this is already part of standard FW codepath)
>
>> BTW, another option might be to "type_of_loader = (9 << 4) | 0", which
>> is what init_pvh_bootparams() does. In fact, whatever is done in the
>> firmware should probably match what that routine does.
> So it wouldn't be possible any longer to tell whether the kernel has
> been booted directly or via grub. I don't like this. The loader type
> is accessible via sysfs after all.
I didn't know that. What is the path?
-boris
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: Xen PVH support in grub2
2017-11-06 16:42 ` Boris Ostrovsky
@ 2017-11-07 7:42 ` Juergen Gross
2017-11-07 7:42 ` [Xen-devel] " Juergen Gross
1 sibling, 0 replies; 73+ messages in thread
From: Juergen Gross @ 2017-11-07 7:42 UTC (permalink / raw)
To: Boris Ostrovsky, Roger Pau Monné
Cc: The development of GNU GRUB, Daniel Kiper, Konrad Rzeszutek Wilk,
xen-devel
On 06/11/17 17:42, Boris Ostrovsky wrote:
> On 11/06/2017 10:05 AM, Juergen Gross wrote:
>> On 06/11/17 15:51, Boris Ostrovsky wrote:
>>> On 11/06/2017 02:16 AM, Juergen Gross wrote:
>>>> On 03/11/17 20:00, Boris Ostrovsky wrote:
>>>>> On 11/03/2017 02:40 PM, Juergen Gross wrote:
>>>>>> On 03/11/17 19:35, Boris Ostrovsky wrote:
>>>>>>> On 11/03/2017 02:23 PM, Juergen Gross wrote:
>>>>>>>> On 03/11/17 19:19, Boris Ostrovsky wrote:
>>>>>>>>> On 11/03/2017 02:05 PM, Juergen Gross wrote:
>>>>>>>>>> So again the question: how to tell whether we are PVH or HVM in
>>>>>>>>>> init_hypervisor_platform()? ACPi tables are scanned way later...
>>>>>>>>> Can we make grub/OVMF append a boot option?
>>>>>>>>>
>>>>>>>>> Or set setup_header.hardware_subarch to something? We already have
>>>>>>>>> X86_SUBARCH_XEN but it is only used by PV. Or we might be able to use
>>>>>>>>> hardware_subarch_data (will need to get a buy-in from x86 maintainers, I
>>>>>>>>> think).
>>>>>>>> But wouldn't this break the idea to reuse the native boot paths in
>>>>>>>> grub/OVMF without further modifications?
>>>>>>> WDYM? We will have to have some sort of a plugin in either one to build
>>>>>>> the zeropage anyway. So we'd set hardware_subarch there, in addition to
>>>>>>> other things like setting memory and such.
>>>>>> But isn't the zeropage already being built? I admit that setting subarch
>>>>>> isn't a big deal, but using another entry with a passed-through pvh
>>>>>> start struct isn't either...
>>>>> I don't follow, sorry. My understanding is that zeropage will be built
>>>>> by PVH-enlightened grub so part of this process would be setting the
>>>>> subarch bit.
>>>> My reasoning was based on Roger's remark:
>>>>
>>>> "OTOH if Linux is capable of booting from the native entry point inside
>>>> of a PVH container, we would only have to port OVMF and grub in order
>>>> to work inside of a PVH container, leaving the rest of the logic
>>>> untouched."
>>> Right, and in my mind porting OVMF/grub includes creating proper zeropage.
>> Aah, okay. I reasoned on the assumption to just enable OVMF/grub to run
>> in PVH environment without touching the parts setting up anything for
>> the new kernel.
>
> Someone needs to do what xen_prepare_pvh() does.
As the loader is filling in the memory map information the only thing
remaining would be setting xen_pvh. And this could be delayed as my test
have shown, so we only need to detect the PVH case from inside the
kernel. One possibility would be the flags in the ACPI FADT table as
Roger mentioned, another idea would be a flag in zeropage set by the
loader.
> And, for 64-bit, we also may need to build early pagetables since
> startup_64() (unlike startup_32()) expects paging to be on. (I don't
> know whether this is already part of standard FW codepath)
This would be done the same way as for a native kernel.
>>> BTW, another option might be to "type_of_loader = (9 << 4) | 0", which
>>> is what init_pvh_bootparams() does. In fact, whatever is done in the
>>> firmware should probably match what that routine does.
>> So it wouldn't be possible any longer to tell whether the kernel has
>> been booted directly or via grub. I don't like this. The loader type
>> is accessible via sysfs after all.
>
> I didn't know that. What is the path?
/proc/sys/kernel/bootloader_type
Juergen
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: [Xen-devel] Xen PVH support in grub2
2017-11-06 16:42 ` Boris Ostrovsky
2017-11-07 7:42 ` Juergen Gross
@ 2017-11-07 7:42 ` Juergen Gross
2017-11-07 16:10 ` Boris Ostrovsky
2017-11-07 16:10 ` [Xen-devel] " Boris Ostrovsky
1 sibling, 2 replies; 73+ messages in thread
From: Juergen Gross @ 2017-11-07 7:42 UTC (permalink / raw)
To: Boris Ostrovsky, Roger Pau Monné
Cc: The development of GNU GRUB, Daniel Kiper, xen-devel,
Konrad Rzeszutek Wilk
On 06/11/17 17:42, Boris Ostrovsky wrote:
> On 11/06/2017 10:05 AM, Juergen Gross wrote:
>> On 06/11/17 15:51, Boris Ostrovsky wrote:
>>> On 11/06/2017 02:16 AM, Juergen Gross wrote:
>>>> On 03/11/17 20:00, Boris Ostrovsky wrote:
>>>>> On 11/03/2017 02:40 PM, Juergen Gross wrote:
>>>>>> On 03/11/17 19:35, Boris Ostrovsky wrote:
>>>>>>> On 11/03/2017 02:23 PM, Juergen Gross wrote:
>>>>>>>> On 03/11/17 19:19, Boris Ostrovsky wrote:
>>>>>>>>> On 11/03/2017 02:05 PM, Juergen Gross wrote:
>>>>>>>>>> So again the question: how to tell whether we are PVH or HVM in
>>>>>>>>>> init_hypervisor_platform()? ACPi tables are scanned way later...
>>>>>>>>> Can we make grub/OVMF append a boot option?
>>>>>>>>>
>>>>>>>>> Or set setup_header.hardware_subarch to something? We already have
>>>>>>>>> X86_SUBARCH_XEN but it is only used by PV. Or we might be able to use
>>>>>>>>> hardware_subarch_data (will need to get a buy-in from x86 maintainers, I
>>>>>>>>> think).
>>>>>>>> But wouldn't this break the idea to reuse the native boot paths in
>>>>>>>> grub/OVMF without further modifications?
>>>>>>> WDYM? We will have to have some sort of a plugin in either one to build
>>>>>>> the zeropage anyway. So we'd set hardware_subarch there, in addition to
>>>>>>> other things like setting memory and such.
>>>>>> But isn't the zeropage already being built? I admit that setting subarch
>>>>>> isn't a big deal, but using another entry with a passed-through pvh
>>>>>> start struct isn't either...
>>>>> I don't follow, sorry. My understanding is that zeropage will be built
>>>>> by PVH-enlightened grub so part of this process would be setting the
>>>>> subarch bit.
>>>> My reasoning was based on Roger's remark:
>>>>
>>>> "OTOH if Linux is capable of booting from the native entry point inside
>>>> of a PVH container, we would only have to port OVMF and grub in order
>>>> to work inside of a PVH container, leaving the rest of the logic
>>>> untouched."
>>> Right, and in my mind porting OVMF/grub includes creating proper zeropage.
>> Aah, okay. I reasoned on the assumption to just enable OVMF/grub to run
>> in PVH environment without touching the parts setting up anything for
>> the new kernel.
>
> Someone needs to do what xen_prepare_pvh() does.
As the loader is filling in the memory map information the only thing
remaining would be setting xen_pvh. And this could be delayed as my test
have shown, so we only need to detect the PVH case from inside the
kernel. One possibility would be the flags in the ACPI FADT table as
Roger mentioned, another idea would be a flag in zeropage set by the
loader.
> And, for 64-bit, we also may need to build early pagetables since
> startup_64() (unlike startup_32()) expects paging to be on. (I don't
> know whether this is already part of standard FW codepath)
This would be done the same way as for a native kernel.
>>> BTW, another option might be to "type_of_loader = (9 << 4) | 0", which
>>> is what init_pvh_bootparams() does. In fact, whatever is done in the
>>> firmware should probably match what that routine does.
>> So it wouldn't be possible any longer to tell whether the kernel has
>> been booted directly or via grub. I don't like this. The loader type
>> is accessible via sysfs after all.
>
> I didn't know that. What is the path?
/proc/sys/kernel/bootloader_type
Juergen
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: Xen PVH support in grub2
2017-11-07 7:42 ` [Xen-devel] " Juergen Gross
@ 2017-11-07 16:10 ` Boris Ostrovsky
2017-11-07 16:10 ` [Xen-devel] " Boris Ostrovsky
1 sibling, 0 replies; 73+ messages in thread
From: Boris Ostrovsky @ 2017-11-07 16:10 UTC (permalink / raw)
To: Juergen Gross, Roger Pau Monné
Cc: The development of GNU GRUB, Daniel Kiper, Konrad Rzeszutek Wilk,
xen-devel
On 11/07/2017 02:42 AM, Juergen Gross wrote:
> On 06/11/17 17:42, Boris Ostrovsky wrote:
>> On 11/06/2017 10:05 AM, Juergen Gross wrote:
>>> On 06/11/17 15:51, Boris Ostrovsky wrote:
>>>> On 11/06/2017 02:16 AM, Juergen Gross wrote:
>>>>> On 03/11/17 20:00, Boris Ostrovsky wrote:
>>>>>> On 11/03/2017 02:40 PM, Juergen Gross wrote:
>>>>>>> On 03/11/17 19:35, Boris Ostrovsky wrote:
>>>>>>>> On 11/03/2017 02:23 PM, Juergen Gross wrote:
>>>>>>>>> On 03/11/17 19:19, Boris Ostrovsky wrote:
>>>>>>>>>> On 11/03/2017 02:05 PM, Juergen Gross wrote:
>>>>>>>>>>> So again the question: how to tell whether we are PVH or HVM in
>>>>>>>>>>> init_hypervisor_platform()? ACPi tables are scanned way later...
>>>>>>>>>> Can we make grub/OVMF append a boot option?
>>>>>>>>>>
>>>>>>>>>> Or set setup_header.hardware_subarch to something? We already have
>>>>>>>>>> X86_SUBARCH_XEN but it is only used by PV. Or we might be able to use
>>>>>>>>>> hardware_subarch_data (will need to get a buy-in from x86 maintainers, I
>>>>>>>>>> think).
>>>>>>>>> But wouldn't this break the idea to reuse the native boot paths in
>>>>>>>>> grub/OVMF without further modifications?
>>>>>>>> WDYM? We will have to have some sort of a plugin in either one to build
>>>>>>>> the zeropage anyway. So we'd set hardware_subarch there, in addition to
>>>>>>>> other things like setting memory and such.
>>>>>>> But isn't the zeropage already being built? I admit that setting subarch
>>>>>>> isn't a big deal, but using another entry with a passed-through pvh
>>>>>>> start struct isn't either...
>>>>>> I don't follow, sorry. My understanding is that zeropage will be built
>>>>>> by PVH-enlightened grub so part of this process would be setting the
>>>>>> subarch bit.
>>>>> My reasoning was based on Roger's remark:
>>>>>
>>>>> "OTOH if Linux is capable of booting from the native entry point inside
>>>>> of a PVH container, we would only have to port OVMF and grub in order
>>>>> to work inside of a PVH container, leaving the rest of the logic
>>>>> untouched."
>>>> Right, and in my mind porting OVMF/grub includes creating proper zeropage.
>>> Aah, okay. I reasoned on the assumption to just enable OVMF/grub to run
>>> in PVH environment without touching the parts setting up anything for
>>> the new kernel.
>> Someone needs to do what xen_prepare_pvh() does.
> As the loader is filling in the memory map information
That's the thing that I thought may need to be done by us (setting
commandline too) . But I haven't looked at Xen support in grub so maybe
it's already there.
> the only thing
> remaining would be setting xen_pvh. And this could be delayed as my test
> have shown, so we only need to detect the PVH case from inside the
> kernel. One possibility would be the flags in the ACPI FADT table as
> Roger mentioned, another idea would be a flag in zeropage set by the
> loader.
>
>> And, for 64-bit, we also may need to build early pagetables since
>> startup_64() (unlike startup_32()) expects paging to be on. (I don't
>> know whether this is already part of standard FW codepath)
> This would be done the same way as for a native kernel.
This is done by Linux trampoline code. AFAIK grub loads kernel in realmode.
-boris
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: [Xen-devel] Xen PVH support in grub2
2017-11-07 7:42 ` [Xen-devel] " Juergen Gross
2017-11-07 16:10 ` Boris Ostrovsky
@ 2017-11-07 16:10 ` Boris Ostrovsky
2017-11-09 5:01 ` Konrad Rzeszutek Wilk
2017-11-09 5:01 ` [Xen-devel] " Konrad Rzeszutek Wilk
1 sibling, 2 replies; 73+ messages in thread
From: Boris Ostrovsky @ 2017-11-07 16:10 UTC (permalink / raw)
To: Juergen Gross, Roger Pau Monné
Cc: The development of GNU GRUB, Daniel Kiper, xen-devel,
Konrad Rzeszutek Wilk
On 11/07/2017 02:42 AM, Juergen Gross wrote:
> On 06/11/17 17:42, Boris Ostrovsky wrote:
>> On 11/06/2017 10:05 AM, Juergen Gross wrote:
>>> On 06/11/17 15:51, Boris Ostrovsky wrote:
>>>> On 11/06/2017 02:16 AM, Juergen Gross wrote:
>>>>> On 03/11/17 20:00, Boris Ostrovsky wrote:
>>>>>> On 11/03/2017 02:40 PM, Juergen Gross wrote:
>>>>>>> On 03/11/17 19:35, Boris Ostrovsky wrote:
>>>>>>>> On 11/03/2017 02:23 PM, Juergen Gross wrote:
>>>>>>>>> On 03/11/17 19:19, Boris Ostrovsky wrote:
>>>>>>>>>> On 11/03/2017 02:05 PM, Juergen Gross wrote:
>>>>>>>>>>> So again the question: how to tell whether we are PVH or HVM in
>>>>>>>>>>> init_hypervisor_platform()? ACPi tables are scanned way later...
>>>>>>>>>> Can we make grub/OVMF append a boot option?
>>>>>>>>>>
>>>>>>>>>> Or set setup_header.hardware_subarch to something? We already have
>>>>>>>>>> X86_SUBARCH_XEN but it is only used by PV. Or we might be able to use
>>>>>>>>>> hardware_subarch_data (will need to get a buy-in from x86 maintainers, I
>>>>>>>>>> think).
>>>>>>>>> But wouldn't this break the idea to reuse the native boot paths in
>>>>>>>>> grub/OVMF without further modifications?
>>>>>>>> WDYM? We will have to have some sort of a plugin in either one to build
>>>>>>>> the zeropage anyway. So we'd set hardware_subarch there, in addition to
>>>>>>>> other things like setting memory and such.
>>>>>>> But isn't the zeropage already being built? I admit that setting subarch
>>>>>>> isn't a big deal, but using another entry with a passed-through pvh
>>>>>>> start struct isn't either...
>>>>>> I don't follow, sorry. My understanding is that zeropage will be built
>>>>>> by PVH-enlightened grub so part of this process would be setting the
>>>>>> subarch bit.
>>>>> My reasoning was based on Roger's remark:
>>>>>
>>>>> "OTOH if Linux is capable of booting from the native entry point inside
>>>>> of a PVH container, we would only have to port OVMF and grub in order
>>>>> to work inside of a PVH container, leaving the rest of the logic
>>>>> untouched."
>>>> Right, and in my mind porting OVMF/grub includes creating proper zeropage.
>>> Aah, okay. I reasoned on the assumption to just enable OVMF/grub to run
>>> in PVH environment without touching the parts setting up anything for
>>> the new kernel.
>> Someone needs to do what xen_prepare_pvh() does.
> As the loader is filling in the memory map information
That's the thing that I thought may need to be done by us (setting
commandline too) . But I haven't looked at Xen support in grub so maybe
it's already there.
> the only thing
> remaining would be setting xen_pvh. And this could be delayed as my test
> have shown, so we only need to detect the PVH case from inside the
> kernel. One possibility would be the flags in the ACPI FADT table as
> Roger mentioned, another idea would be a flag in zeropage set by the
> loader.
>
>> And, for 64-bit, we also may need to build early pagetables since
>> startup_64() (unlike startup_32()) expects paging to be on. (I don't
>> know whether this is already part of standard FW codepath)
> This would be done the same way as for a native kernel.
This is done by Linux trampoline code. AFAIK grub loads kernel in realmode.
-boris
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: Xen PVH support in grub2
2017-11-07 16:10 ` [Xen-devel] " Boris Ostrovsky
@ 2017-11-09 5:01 ` Konrad Rzeszutek Wilk
2017-11-09 5:01 ` [Xen-devel] " Konrad Rzeszutek Wilk
1 sibling, 0 replies; 73+ messages in thread
From: Konrad Rzeszutek Wilk @ 2017-11-09 5:01 UTC (permalink / raw)
To: Boris Ostrovsky, maran.wilson
Cc: Juergen Gross, The development of GNU GRUB, Daniel Kiper,
xen-devel, Roger Pau Monné
On Tue, Nov 07, 2017 at 11:10:29AM -0500, Boris Ostrovsky wrote:
> On 11/07/2017 02:42 AM, Juergen Gross wrote:
> > On 06/11/17 17:42, Boris Ostrovsky wrote:
> >> On 11/06/2017 10:05 AM, Juergen Gross wrote:
> >>> On 06/11/17 15:51, Boris Ostrovsky wrote:
> >>>> On 11/06/2017 02:16 AM, Juergen Gross wrote:
> >>>>> On 03/11/17 20:00, Boris Ostrovsky wrote:
> >>>>>> On 11/03/2017 02:40 PM, Juergen Gross wrote:
> >>>>>>> On 03/11/17 19:35, Boris Ostrovsky wrote:
> >>>>>>>> On 11/03/2017 02:23 PM, Juergen Gross wrote:
> >>>>>>>>> On 03/11/17 19:19, Boris Ostrovsky wrote:
> >>>>>>>>>> On 11/03/2017 02:05 PM, Juergen Gross wrote:
> >>>>>>>>>>> So again the question: how to tell whether we are PVH or HVM in
> >>>>>>>>>>> init_hypervisor_platform()? ACPi tables are scanned way later...
> >>>>>>>>>> Can we make grub/OVMF append a boot option?
> >>>>>>>>>>
> >>>>>>>>>> Or set setup_header.hardware_subarch to something? We already have
> >>>>>>>>>> X86_SUBARCH_XEN but it is only used by PV. Or we might be able to use
> >>>>>>>>>> hardware_subarch_data (will need to get a buy-in from x86 maintainers, I
> >>>>>>>>>> think).
> >>>>>>>>> But wouldn't this break the idea to reuse the native boot paths in
> >>>>>>>>> grub/OVMF without further modifications?
> >>>>>>>> WDYM? We will have to have some sort of a plugin in either one to build
> >>>>>>>> the zeropage anyway. So we'd set hardware_subarch there, in addition to
> >>>>>>>> other things like setting memory and such.
> >>>>>>> But isn't the zeropage already being built? I admit that setting subarch
> >>>>>>> isn't a big deal, but using another entry with a passed-through pvh
> >>>>>>> start struct isn't either...
> >>>>>> I don't follow, sorry. My understanding is that zeropage will be built
> >>>>>> by PVH-enlightened grub so part of this process would be setting the
> >>>>>> subarch bit.
> >>>>> My reasoning was based on Roger's remark:
> >>>>>
> >>>>> "OTOH if Linux is capable of booting from the native entry point inside
> >>>>> of a PVH container, we would only have to port OVMF and grub in order
> >>>>> to work inside of a PVH container, leaving the rest of the logic
> >>>>> untouched."
> >>>> Right, and in my mind porting OVMF/grub includes creating proper zeropage.
> >>> Aah, okay. I reasoned on the assumption to just enable OVMF/grub to run
> >>> in PVH environment without touching the parts setting up anything for
> >>> the new kernel.
> >> Someone needs to do what xen_prepare_pvh() does.
> > As the loader is filling in the memory map information
>
> That's the thing that I thought may need to be done by us (setting
> commandline too) . But I haven't looked at Xen support in grub so maybe
> it's already there.
>
> > the only thing
> > remaining would be setting xen_pvh. And this could be delayed as my test
> > have shown, so we only need to detect the PVH case from inside the
> > kernel. One possibility would be the flags in the ACPI FADT table as
> > Roger mentioned, another idea would be a flag in zeropage set by the
> > loader.
> >
> >> And, for 64-bit, we also may need to build early pagetables since
> >> startup_64() (unlike startup_32()) expects paging to be on. (I don't
> >> know whether this is already part of standard FW codepath)
> > This would be done the same way as for a native kernel.
>
> This is done by Linux trampoline code. AFAIK grub loads kernel in realmode.
Adding in Maran who is looking at re-using PVH for qemu launching
the kernels.
>
>
> -boris
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> https://lists.xen.org/xen-devel
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: [Xen-devel] Xen PVH support in grub2
2017-11-07 16:10 ` [Xen-devel] " Boris Ostrovsky
2017-11-09 5:01 ` Konrad Rzeszutek Wilk
@ 2017-11-09 5:01 ` Konrad Rzeszutek Wilk
1 sibling, 0 replies; 73+ messages in thread
From: Konrad Rzeszutek Wilk @ 2017-11-09 5:01 UTC (permalink / raw)
To: Boris Ostrovsky, maran.wilson
Cc: Juergen Gross, Roger Pau Monné,
The development of GNU GRUB, Daniel Kiper, xen-devel
On Tue, Nov 07, 2017 at 11:10:29AM -0500, Boris Ostrovsky wrote:
> On 11/07/2017 02:42 AM, Juergen Gross wrote:
> > On 06/11/17 17:42, Boris Ostrovsky wrote:
> >> On 11/06/2017 10:05 AM, Juergen Gross wrote:
> >>> On 06/11/17 15:51, Boris Ostrovsky wrote:
> >>>> On 11/06/2017 02:16 AM, Juergen Gross wrote:
> >>>>> On 03/11/17 20:00, Boris Ostrovsky wrote:
> >>>>>> On 11/03/2017 02:40 PM, Juergen Gross wrote:
> >>>>>>> On 03/11/17 19:35, Boris Ostrovsky wrote:
> >>>>>>>> On 11/03/2017 02:23 PM, Juergen Gross wrote:
> >>>>>>>>> On 03/11/17 19:19, Boris Ostrovsky wrote:
> >>>>>>>>>> On 11/03/2017 02:05 PM, Juergen Gross wrote:
> >>>>>>>>>>> So again the question: how to tell whether we are PVH or HVM in
> >>>>>>>>>>> init_hypervisor_platform()? ACPi tables are scanned way later...
> >>>>>>>>>> Can we make grub/OVMF append a boot option?
> >>>>>>>>>>
> >>>>>>>>>> Or set setup_header.hardware_subarch to something? We already have
> >>>>>>>>>> X86_SUBARCH_XEN but it is only used by PV. Or we might be able to use
> >>>>>>>>>> hardware_subarch_data (will need to get a buy-in from x86 maintainers, I
> >>>>>>>>>> think).
> >>>>>>>>> But wouldn't this break the idea to reuse the native boot paths in
> >>>>>>>>> grub/OVMF without further modifications?
> >>>>>>>> WDYM? We will have to have some sort of a plugin in either one to build
> >>>>>>>> the zeropage anyway. So we'd set hardware_subarch there, in addition to
> >>>>>>>> other things like setting memory and such.
> >>>>>>> But isn't the zeropage already being built? I admit that setting subarch
> >>>>>>> isn't a big deal, but using another entry with a passed-through pvh
> >>>>>>> start struct isn't either...
> >>>>>> I don't follow, sorry. My understanding is that zeropage will be built
> >>>>>> by PVH-enlightened grub so part of this process would be setting the
> >>>>>> subarch bit.
> >>>>> My reasoning was based on Roger's remark:
> >>>>>
> >>>>> "OTOH if Linux is capable of booting from the native entry point inside
> >>>>> of a PVH container, we would only have to port OVMF and grub in order
> >>>>> to work inside of a PVH container, leaving the rest of the logic
> >>>>> untouched."
> >>>> Right, and in my mind porting OVMF/grub includes creating proper zeropage.
> >>> Aah, okay. I reasoned on the assumption to just enable OVMF/grub to run
> >>> in PVH environment without touching the parts setting up anything for
> >>> the new kernel.
> >> Someone needs to do what xen_prepare_pvh() does.
> > As the loader is filling in the memory map information
>
> That's the thing that I thought may need to be done by us (setting
> commandline too) . But I haven't looked at Xen support in grub so maybe
> it's already there.
>
> > the only thing
> > remaining would be setting xen_pvh. And this could be delayed as my test
> > have shown, so we only need to detect the PVH case from inside the
> > kernel. One possibility would be the flags in the ACPI FADT table as
> > Roger mentioned, another idea would be a flag in zeropage set by the
> > loader.
> >
> >> And, for 64-bit, we also may need to build early pagetables since
> >> startup_64() (unlike startup_32()) expects paging to be on. (I don't
> >> know whether this is already part of standard FW codepath)
> > This would be done the same way as for a native kernel.
>
> This is done by Linux trampoline code. AFAIK grub loads kernel in realmode.
Adding in Maran who is looking at re-using PVH for qemu launching
the kernels.
>
>
> -boris
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: Xen PVH support in grub2
2017-11-06 15:05 ` Juergen Gross
2017-11-06 16:42 ` Boris Ostrovsky
@ 2017-11-06 16:42 ` Boris Ostrovsky
1 sibling, 0 replies; 73+ messages in thread
From: Boris Ostrovsky @ 2017-11-06 16:42 UTC (permalink / raw)
To: Juergen Gross, Roger Pau Monné
Cc: The development of GNU GRUB, Daniel Kiper, Konrad Rzeszutek Wilk,
xen-devel
On 11/06/2017 10:05 AM, Juergen Gross wrote:
> On 06/11/17 15:51, Boris Ostrovsky wrote:
>> On 11/06/2017 02:16 AM, Juergen Gross wrote:
>>> On 03/11/17 20:00, Boris Ostrovsky wrote:
>>>> On 11/03/2017 02:40 PM, Juergen Gross wrote:
>>>>> On 03/11/17 19:35, Boris Ostrovsky wrote:
>>>>>> On 11/03/2017 02:23 PM, Juergen Gross wrote:
>>>>>>> On 03/11/17 19:19, Boris Ostrovsky wrote:
>>>>>>>> On 11/03/2017 02:05 PM, Juergen Gross wrote:
>>>>>>>>> So again the question: how to tell whether we are PVH or HVM in
>>>>>>>>> init_hypervisor_platform()? ACPi tables are scanned way later...
>>>>>>>> Can we make grub/OVMF append a boot option?
>>>>>>>>
>>>>>>>> Or set setup_header.hardware_subarch to something? We already have
>>>>>>>> X86_SUBARCH_XEN but it is only used by PV. Or we might be able to use
>>>>>>>> hardware_subarch_data (will need to get a buy-in from x86 maintainers, I
>>>>>>>> think).
>>>>>>> But wouldn't this break the idea to reuse the native boot paths in
>>>>>>> grub/OVMF without further modifications?
>>>>>> WDYM? We will have to have some sort of a plugin in either one to build
>>>>>> the zeropage anyway. So we'd set hardware_subarch there, in addition to
>>>>>> other things like setting memory and such.
>>>>> But isn't the zeropage already being built? I admit that setting subarch
>>>>> isn't a big deal, but using another entry with a passed-through pvh
>>>>> start struct isn't either...
>>>> I don't follow, sorry. My understanding is that zeropage will be built
>>>> by PVH-enlightened grub so part of this process would be setting the
>>>> subarch bit.
>>> My reasoning was based on Roger's remark:
>>>
>>> "OTOH if Linux is capable of booting from the native entry point inside
>>> of a PVH container, we would only have to port OVMF and grub in order
>>> to work inside of a PVH container, leaving the rest of the logic
>>> untouched."
>> Right, and in my mind porting OVMF/grub includes creating proper zeropage.
> Aah, okay. I reasoned on the assumption to just enable OVMF/grub to run
> in PVH environment without touching the parts setting up anything for
> the new kernel.
Someone needs to do what xen_prepare_pvh() does.
And, for 64-bit, we also may need to build early pagetables since
startup_64() (unlike startup_32()) expects paging to be on. (I don't
know whether this is already part of standard FW codepath)
>
>> BTW, another option might be to "type_of_loader = (9 << 4) | 0", which
>> is what init_pvh_bootparams() does. In fact, whatever is done in the
>> firmware should probably match what that routine does.
> So it wouldn't be possible any longer to tell whether the kernel has
> been booted directly or via grub. I don't like this. The loader type
> is accessible via sysfs after all.
I didn't know that. What is the path?
-boris
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: Xen PVH support in grub2
2017-11-06 14:51 ` [Xen-devel] " Boris Ostrovsky
2017-11-06 15:05 ` Juergen Gross
@ 2017-11-06 15:05 ` Juergen Gross
1 sibling, 0 replies; 73+ messages in thread
From: Juergen Gross @ 2017-11-06 15:05 UTC (permalink / raw)
To: Boris Ostrovsky, Roger Pau Monné
Cc: The development of GNU GRUB, Daniel Kiper, xen-devel
On 06/11/17 15:51, Boris Ostrovsky wrote:
> On 11/06/2017 02:16 AM, Juergen Gross wrote:
>> On 03/11/17 20:00, Boris Ostrovsky wrote:
>>> On 11/03/2017 02:40 PM, Juergen Gross wrote:
>>>> On 03/11/17 19:35, Boris Ostrovsky wrote:
>>>>> On 11/03/2017 02:23 PM, Juergen Gross wrote:
>>>>>> On 03/11/17 19:19, Boris Ostrovsky wrote:
>>>>>>> On 11/03/2017 02:05 PM, Juergen Gross wrote:
>>>>>>>> So again the question: how to tell whether we are PVH or HVM in
>>>>>>>> init_hypervisor_platform()? ACPi tables are scanned way later...
>>>>>>> Can we make grub/OVMF append a boot option?
>>>>>>>
>>>>>>> Or set setup_header.hardware_subarch to something? We already have
>>>>>>> X86_SUBARCH_XEN but it is only used by PV. Or we might be able to use
>>>>>>> hardware_subarch_data (will need to get a buy-in from x86 maintainers, I
>>>>>>> think).
>>>>>> But wouldn't this break the idea to reuse the native boot paths in
>>>>>> grub/OVMF without further modifications?
>>>>> WDYM? We will have to have some sort of a plugin in either one to build
>>>>> the zeropage anyway. So we'd set hardware_subarch there, in addition to
>>>>> other things like setting memory and such.
>>>> But isn't the zeropage already being built? I admit that setting subarch
>>>> isn't a big deal, but using another entry with a passed-through pvh
>>>> start struct isn't either...
>>> I don't follow, sorry. My understanding is that zeropage will be built
>>> by PVH-enlightened grub so part of this process would be setting the
>>> subarch bit.
>> My reasoning was based on Roger's remark:
>>
>> "OTOH if Linux is capable of booting from the native entry point inside
>> of a PVH container, we would only have to port OVMF and grub in order
>> to work inside of a PVH container, leaving the rest of the logic
>> untouched."
>
> Right, and in my mind porting OVMF/grub includes creating proper zeropage.
Aah, okay. I reasoned on the assumption to just enable OVMF/grub to run
in PVH environment without touching the parts setting up anything for
the new kernel.
> BTW, another option might be to "type_of_loader = (9 << 4) | 0", which
> is what init_pvh_bootparams() does. In fact, whatever is done in the
> firmware should probably match what that routine does.
So it wouldn't be possible any longer to tell whether the kernel has
been booted directly or via grub. I don't like this. The loader type
is accessible via sysfs after all.
Juergen
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: Xen PVH support in grub2
2017-11-03 18:35 ` Boris Ostrovsky
2017-11-03 18:40 ` Juergen Gross
@ 2017-11-03 18:40 ` Juergen Gross
1 sibling, 0 replies; 73+ messages in thread
From: Juergen Gross @ 2017-11-03 18:40 UTC (permalink / raw)
To: Boris Ostrovsky, Roger Pau Monné
Cc: The development of GNU GRUB, Daniel Kiper, xen-devel
On 03/11/17 19:35, Boris Ostrovsky wrote:
> On 11/03/2017 02:23 PM, Juergen Gross wrote:
>> On 03/11/17 19:19, Boris Ostrovsky wrote:
>>> On 11/03/2017 02:05 PM, Juergen Gross wrote:
>>>> So again the question: how to tell whether we are PVH or HVM in
>>>> init_hypervisor_platform()? ACPi tables are scanned way later...
>>> Can we make grub/OVMF append a boot option?
>>>
>>> Or set setup_header.hardware_subarch to something? We already have
>>> X86_SUBARCH_XEN but it is only used by PV. Or we might be able to use
>>> hardware_subarch_data (will need to get a buy-in from x86 maintainers, I
>>> think).
>> But wouldn't this break the idea to reuse the native boot paths in
>> grub/OVMF without further modifications?
>
> WDYM? We will have to have some sort of a plugin in either one to build
> the zeropage anyway. So we'd set hardware_subarch there, in addition to
> other things like setting memory and such.
But isn't the zeropage already being built? I admit that setting subarch
isn't a big deal, but using another entry with a passed-through pvh
start struct isn't either...
Juergen
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: Xen PVH support in grub2
2017-11-03 18:23 ` Juergen Gross
2017-11-03 18:35 ` Boris Ostrovsky
@ 2017-11-03 18:35 ` Boris Ostrovsky
2017-11-03 18:37 ` [Xen-devel] " Roger Pau Monné
2 siblings, 0 replies; 73+ messages in thread
From: Boris Ostrovsky @ 2017-11-03 18:35 UTC (permalink / raw)
To: Juergen Gross, Roger Pau Monné
Cc: The development of GNU GRUB, Daniel Kiper, xen-devel
On 11/03/2017 02:23 PM, Juergen Gross wrote:
> On 03/11/17 19:19, Boris Ostrovsky wrote:
>> On 11/03/2017 02:05 PM, Juergen Gross wrote:
>>> So again the question: how to tell whether we are PVH or HVM in
>>> init_hypervisor_platform()? ACPi tables are scanned way later...
>> Can we make grub/OVMF append a boot option?
>>
>> Or set setup_header.hardware_subarch to something? We already have
>> X86_SUBARCH_XEN but it is only used by PV. Or we might be able to use
>> hardware_subarch_data (will need to get a buy-in from x86 maintainers, I
>> think).
> But wouldn't this break the idea to reuse the native boot paths in
> grub/OVMF without further modifications?
WDYM? We will have to have some sort of a plugin in either one to build
the zeropage anyway. So we'd set hardware_subarch there, in addition to
other things like setting memory and such.
-boris
>
> I'd rather have a way to ask the hypervisor whether we are in PVH mode
> (e.g. via CPUID or a hypercall to test for a devicemodel having itself
> registered).
>
> In case we need to set an indicator in zeropage I'd opt for reusing
> X86_SUBARCH_XEN.
>
>
> Juergen
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: Xen PVH support in grub2
2017-11-03 18:23 ` Juergen Gross
@ 2017-11-03 18:37 ` Roger Pau Monné
2017-11-03 18:35 ` Boris Ostrovsky
2017-11-03 18:37 ` [Xen-devel] " Roger Pau Monné
2 siblings, 0 replies; 73+ messages in thread
From: Roger Pau Monné @ 2017-11-03 18:37 UTC (permalink / raw)
To: Juergen Gross
Cc: The development of GNU GRUB, Boris Ostrovsky, Daniel Kiper, xen-devel
On Fri, Nov 03, 2017 at 07:23:50PM +0100, Juergen Gross wrote:
> On 03/11/17 19:19, Boris Ostrovsky wrote:
> > On 11/03/2017 02:05 PM, Juergen Gross wrote:
> >>
> >> So again the question: how to tell whether we are PVH or HVM in
> >> init_hypervisor_platform()? ACPi tables are scanned way later...
> >
> > Can we make grub/OVMF append a boot option?
> >
> > Or set setup_header.hardware_subarch to something? We already have
> > X86_SUBARCH_XEN but it is only used by PV. Or we might be able to use
> > hardware_subarch_data (will need to get a buy-in from x86 maintainers, I
> > think).
>
> But wouldn't this break the idea to reuse the native boot paths in
> grub/OVMF without further modifications?
>
> I'd rather have a way to ask the hypervisor whether we are in PVH mode
> (e.g. via CPUID or a hypercall to test for a devicemodel having itself
> registered).
Keep in mind that from Xen's PoV PVH is just a HVM guest. Xen
currently keeps a bitmap of the emulated devices that are available to
a guest, but that's so far an internal field. We could consider
exporting this on a cpuid leaf, but then we need to make it a fixed
ABI.
Maybe we can make a list of platform devices that are not available on
PVH and that Linux assumes to be present?
Roger.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: [Xen-devel] Xen PVH support in grub2
@ 2017-11-03 18:37 ` Roger Pau Monné
0 siblings, 0 replies; 73+ messages in thread
From: Roger Pau Monné @ 2017-11-03 18:37 UTC (permalink / raw)
To: Juergen Gross
Cc: Boris Ostrovsky, The development of GNU GRUB, Daniel Kiper, xen-devel
On Fri, Nov 03, 2017 at 07:23:50PM +0100, Juergen Gross wrote:
> On 03/11/17 19:19, Boris Ostrovsky wrote:
> > On 11/03/2017 02:05 PM, Juergen Gross wrote:
> >>
> >> So again the question: how to tell whether we are PVH or HVM in
> >> init_hypervisor_platform()? ACPi tables are scanned way later...
> >
> > Can we make grub/OVMF append a boot option?
> >
> > Or set setup_header.hardware_subarch to something? We already have
> > X86_SUBARCH_XEN but it is only used by PV. Or we might be able to use
> > hardware_subarch_data (will need to get a buy-in from x86 maintainers, I
> > think).
>
> But wouldn't this break the idea to reuse the native boot paths in
> grub/OVMF without further modifications?
>
> I'd rather have a way to ask the hypervisor whether we are in PVH mode
> (e.g. via CPUID or a hypercall to test for a devicemodel having itself
> registered).
Keep in mind that from Xen's PoV PVH is just a HVM guest. Xen
currently keeps a bitmap of the emulated devices that are available to
a guest, but that's so far an internal field. We could consider
exporting this on a cpuid leaf, but then we need to make it a fixed
ABI.
Maybe we can make a list of platform devices that are not available on
PVH and that Linux assumes to be present?
Roger.
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: [Xen-devel] Xen PVH support in grub2
2017-11-03 18:37 ` [Xen-devel] " Roger Pau Monné
(?)
@ 2017-11-03 18:47 ` Juergen Gross
-1 siblings, 0 replies; 73+ messages in thread
From: Juergen Gross @ 2017-11-03 18:47 UTC (permalink / raw)
To: Roger Pau Monné
Cc: Boris Ostrovsky, The development of GNU GRUB, Daniel Kiper, xen-devel
On 03/11/17 19:37, Roger Pau Monné wrote:
> On Fri, Nov 03, 2017 at 07:23:50PM +0100, Juergen Gross wrote:
>> On 03/11/17 19:19, Boris Ostrovsky wrote:
>>> On 11/03/2017 02:05 PM, Juergen Gross wrote:
>>>>
>>>> So again the question: how to tell whether we are PVH or HVM in
>>>> init_hypervisor_platform()? ACPi tables are scanned way later...
>>>
>>> Can we make grub/OVMF append a boot option?
>>>
>>> Or set setup_header.hardware_subarch to something? We already have
>>> X86_SUBARCH_XEN but it is only used by PV. Or we might be able to use
>>> hardware_subarch_data (will need to get a buy-in from x86 maintainers, I
>>> think).
>>
>> But wouldn't this break the idea to reuse the native boot paths in
>> grub/OVMF without further modifications?
>>
>> I'd rather have a way to ask the hypervisor whether we are in PVH mode
>> (e.g. via CPUID or a hypercall to test for a devicemodel having itself
>> registered).
>
> Keep in mind that from Xen's PoV PVH is just a HVM guest. Xen
> currently keeps a bitmap of the emulated devices that are available to
> a guest, but that's so far an internal field. We could consider
> exporting this on a cpuid leaf, but then we need to make it a fixed
> ABI.
>
> Maybe we can make a list of platform devices that are not available on
> PVH and that Linux assumes to be present?
The main problem in Linux is that we have to know whether we are a HVM
or a PVH guest. Okay, we could scan for the Xen PCI-device to make the
distinction, but this is again rather late in the boot process.
Using implicit assumptions is normally a bad way to make such a
decision. I'd rather be either told by the boot loader I'm running as
PVH guest (and the boot loader does know it), or I want to ask the
hypervisor (which right now doesn't know), or I need another clear
distinction like the existence of the Xen PCI-device (which might be
a problem in the future for a PVH dom0 in a nested Xen environment).
Juergen
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: Xen PVH support in grub2
2017-11-03 18:37 ` [Xen-devel] " Roger Pau Monné
(?)
(?)
@ 2017-11-03 18:47 ` Juergen Gross
-1 siblings, 0 replies; 73+ messages in thread
From: Juergen Gross @ 2017-11-03 18:47 UTC (permalink / raw)
To: Roger Pau Monné
Cc: The development of GNU GRUB, Boris Ostrovsky, Daniel Kiper, xen-devel
On 03/11/17 19:37, Roger Pau Monné wrote:
> On Fri, Nov 03, 2017 at 07:23:50PM +0100, Juergen Gross wrote:
>> On 03/11/17 19:19, Boris Ostrovsky wrote:
>>> On 11/03/2017 02:05 PM, Juergen Gross wrote:
>>>>
>>>> So again the question: how to tell whether we are PVH or HVM in
>>>> init_hypervisor_platform()? ACPi tables are scanned way later...
>>>
>>> Can we make grub/OVMF append a boot option?
>>>
>>> Or set setup_header.hardware_subarch to something? We already have
>>> X86_SUBARCH_XEN but it is only used by PV. Or we might be able to use
>>> hardware_subarch_data (will need to get a buy-in from x86 maintainers, I
>>> think).
>>
>> But wouldn't this break the idea to reuse the native boot paths in
>> grub/OVMF without further modifications?
>>
>> I'd rather have a way to ask the hypervisor whether we are in PVH mode
>> (e.g. via CPUID or a hypercall to test for a devicemodel having itself
>> registered).
>
> Keep in mind that from Xen's PoV PVH is just a HVM guest. Xen
> currently keeps a bitmap of the emulated devices that are available to
> a guest, but that's so far an internal field. We could consider
> exporting this on a cpuid leaf, but then we need to make it a fixed
> ABI.
>
> Maybe we can make a list of platform devices that are not available on
> PVH and that Linux assumes to be present?
The main problem in Linux is that we have to know whether we are a HVM
or a PVH guest. Okay, we could scan for the Xen PCI-device to make the
distinction, but this is again rather late in the boot process.
Using implicit assumptions is normally a bad way to make such a
decision. I'd rather be either told by the boot loader I'm running as
PVH guest (and the boot loader does know it), or I want to ask the
hypervisor (which right now doesn't know), or I need another clear
distinction like the existence of the Xen PCI-device (which might be
a problem in the future for a PVH dom0 in a nested Xen environment).
Juergen
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: Xen PVH support in grub2
2017-11-03 18:19 ` [Xen-devel] " Boris Ostrovsky
2017-11-03 18:23 ` Juergen Gross
@ 2017-11-03 18:23 ` Juergen Gross
1 sibling, 0 replies; 73+ messages in thread
From: Juergen Gross @ 2017-11-03 18:23 UTC (permalink / raw)
To: Boris Ostrovsky, Roger Pau Monné
Cc: The development of GNU GRUB, Daniel Kiper, xen-devel
On 03/11/17 19:19, Boris Ostrovsky wrote:
> On 11/03/2017 02:05 PM, Juergen Gross wrote:
>>
>> So again the question: how to tell whether we are PVH or HVM in
>> init_hypervisor_platform()? ACPi tables are scanned way later...
>
> Can we make grub/OVMF append a boot option?
>
> Or set setup_header.hardware_subarch to something? We already have
> X86_SUBARCH_XEN but it is only used by PV. Or we might be able to use
> hardware_subarch_data (will need to get a buy-in from x86 maintainers, I
> think).
But wouldn't this break the idea to reuse the native boot paths in
grub/OVMF without further modifications?
I'd rather have a way to ask the hypervisor whether we are in PVH mode
(e.g. via CPUID or a hypercall to test for a devicemodel having itself
registered).
In case we need to set an indicator in zeropage I'd opt for reusing
X86_SUBARCH_XEN.
Juergen
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: Xen PVH support in grub2
2017-11-03 15:27 ` Juergen Gross
2017-11-03 18:05 ` Juergen Gross
@ 2017-11-03 18:05 ` Juergen Gross
1 sibling, 0 replies; 73+ messages in thread
From: Juergen Gross @ 2017-11-03 18:05 UTC (permalink / raw)
To: Boris Ostrovsky, Roger Pau Monné
Cc: The development of GNU GRUB, Daniel Kiper, xen-devel
On 03/11/17 16:27, Juergen Gross wrote:
> On 03/11/17 16:10, Boris Ostrovsky wrote:
>> On 11/03/2017 10:59 AM, Juergen Gross wrote:
>>> On 03/11/17 15:36, Boris Ostrovsky wrote:
>>>> On 11/03/2017 10:24 AM, Juergen Gross wrote:
>>>>> On 03/11/17 15:07, Roger Pau Monné wrote:
>>>>>> On Fri, Nov 03, 2017 at 01:50:11PM +0100, Juergen Gross wrote:
>>>>>>> On 03/11/17 13:17, Roger Pau Monné wrote:
>>>>>>>> On Fri, Nov 03, 2017 at 01:00:46PM +0100, Juergen Gross wrote:
>>>>>>>>> On 29/09/17 17:51, Roger Pau Monné wrote:
>>>>>>>>>> On Fri, Sep 29, 2017 at 03:33:58PM +0000, Juergen Gross wrote:
>>>>>>>>>>> On 29/09/17 17:24, Roger Pau Monné wrote:
>>>>>>>>>>>> On Fri, Sep 29, 2017 at 02:46:53PM +0000, Juergen Gross wrote:
>>>>>>>>>>>> Then, I also wonder whether it would make sense for this grub to load
>>>>>>>>>>>> the kernel using the PVH entry point or the native entry point. Would
>>>>>>>>>>>> it be possible to boot a Linux kernel up to the point where cpuid can
>>>>>>>>>>>> be used inside of a PVH container?
>>>>>>>>>>> I don't think today's Linux allows that. This has been discussed
>>>>>>>>>>> very thoroughly at the time Boris added PVH V2 support to the kernel.
>>>>>>>>>> OK, I'm not going to insist on that, but my plans for FreeBSD is to
>>>>>>>>>> make the native entry point capable of booting inside of a PVH
>>>>>>>>>> container up to the point where cpuid (or whatever method) can be used
>>>>>>>>>> to detect the environment.
>>>>>>>>> Looking more thoroughly into the Linux boot code I think this could
>>>>>>>>> work for Linux, too. But only if we can tell PVH from HVM in the guest.
>>>>>>>>> How would you do that in FreeBSD? Via flags in the boot params? This
>>>>>>>>> would the have to be done in the boot loader (e.g. grub or OVMF).
>>>>>>>> My plan was not to differentiate between HVM and PVH, but rather to
>>>>>>>> make use of the ACPI information in order to decide which devices are
>>>>>>>> available and which are not inside of a PVH guest.
>>>>>>>>
>>>>>>>> For example in the FADT "IA-PC Boot Architecture Flags" field for PVH
>>>>>>>> we already set "VGA Not Present" and "CMOS RTC Not Present". There
>>>>>>>> might be other flags/fields that must be set, but I would like to
>>>>>>>> avoid having a CPUID bit or similar saying "PVH", because then Xen
>>>>>>>> will be tied to always providing the same set of devices in PVH
>>>>>>>> containers.
>>>>>>> Why? This would depend on the semantics tied to the flag. It could just
>>>>>>> mean "don't assume availability of legacy stuff" (e.g. BIOS calls).
>>>>>>>
>>>>>>> Linux would have a problem with the ACPI approach as it would try BIOS
>>>>>>> calls way before it is initializing its ACPI handling. So in Linux I'd
>>>>>>> need another way to tell I'm running in PVH mode, e.g. a "no legacy"
>>>>>>> bit in the Xen HVM cpuid leaf.
>>>>>> If you are booted from the PVH entry point, there's no BIOS or UEFI
>>>>>> (ie: no firmware), if you are booted from the BIOS entry point there's
>>>>>> a BIOS and the same applies to UEFI. How does Linux differentiate
>>>>>> whether it's booted from BIOS or UEFI?
>>>>> They use different entries.
>>>> In fact, we had a discussion with Matt Fleming (Linux EFI maintainer) to
>>>> see if we can use EFI entry point to also be able to boot PVH guest but
>>>> found some issues with that approach, which is why we ended up with a
>>>> dedicated PVH entry point.
>>>>
>>>> I am curious though, Juergen --- what do we need besides zeropage to
>>>> allow us to boot PVH from startup_64?
>>> Oh, you are right. I managed to get lost in the early boot paths.
>>>
>>> Only setting up the hyperpage seems to be missing, but this should be
>>> doable. And setting xen_pvh, of course.
>>
>> That last part was actually my question --- do we need to have xen_pvh
>> set before we get to xen-specific code for the first time (which I think
>> is init_hypervisor_platform()) from startup_64?
>>
>> Because if we do --- who will set it?
>
> This should be easily testable: Just set an internal indicator in
> xen_prepare_pvh() and copy that to xen_pvh in init_hypervisor_platform()
> and see whether the kernel still comes up in PVH mode.
>
> The same can be done with setting up the hypercall page.
>
> Just building a kernel trying that...
Okay, setting xen_pvh late is working. I can't omit setting up the
hypercall_page right now, of course, as then I wouldn't get the memory
map. But this shouldn't be a problem for the grub/OVMF case as those
would setup the memory map for the guest in zeropage.
So again the question: how to tell whether we are PVH or HVM in
init_hypervisor_platform()? ACPi tables are scanned way later...
Juergen
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: Xen PVH support in grub2
2017-11-03 15:10 ` Boris Ostrovsky
2017-11-03 15:27 ` Juergen Gross
@ 2017-11-03 15:27 ` Juergen Gross
1 sibling, 0 replies; 73+ messages in thread
From: Juergen Gross @ 2017-11-03 15:27 UTC (permalink / raw)
To: Boris Ostrovsky, Roger Pau Monné
Cc: The development of GNU GRUB, Daniel Kiper, xen-devel
On 03/11/17 16:10, Boris Ostrovsky wrote:
> On 11/03/2017 10:59 AM, Juergen Gross wrote:
>> On 03/11/17 15:36, Boris Ostrovsky wrote:
>>> On 11/03/2017 10:24 AM, Juergen Gross wrote:
>>>> On 03/11/17 15:07, Roger Pau Monné wrote:
>>>>> On Fri, Nov 03, 2017 at 01:50:11PM +0100, Juergen Gross wrote:
>>>>>> On 03/11/17 13:17, Roger Pau Monné wrote:
>>>>>>> On Fri, Nov 03, 2017 at 01:00:46PM +0100, Juergen Gross wrote:
>>>>>>>> On 29/09/17 17:51, Roger Pau Monné wrote:
>>>>>>>>> On Fri, Sep 29, 2017 at 03:33:58PM +0000, Juergen Gross wrote:
>>>>>>>>>> On 29/09/17 17:24, Roger Pau Monné wrote:
>>>>>>>>>>> On Fri, Sep 29, 2017 at 02:46:53PM +0000, Juergen Gross wrote:
>>>>>>>>>>> Then, I also wonder whether it would make sense for this grub to load
>>>>>>>>>>> the kernel using the PVH entry point or the native entry point. Would
>>>>>>>>>>> it be possible to boot a Linux kernel up to the point where cpuid can
>>>>>>>>>>> be used inside of a PVH container?
>>>>>>>>>> I don't think today's Linux allows that. This has been discussed
>>>>>>>>>> very thoroughly at the time Boris added PVH V2 support to the kernel.
>>>>>>>>> OK, I'm not going to insist on that, but my plans for FreeBSD is to
>>>>>>>>> make the native entry point capable of booting inside of a PVH
>>>>>>>>> container up to the point where cpuid (or whatever method) can be used
>>>>>>>>> to detect the environment.
>>>>>>>> Looking more thoroughly into the Linux boot code I think this could
>>>>>>>> work for Linux, too. But only if we can tell PVH from HVM in the guest.
>>>>>>>> How would you do that in FreeBSD? Via flags in the boot params? This
>>>>>>>> would the have to be done in the boot loader (e.g. grub or OVMF).
>>>>>>> My plan was not to differentiate between HVM and PVH, but rather to
>>>>>>> make use of the ACPI information in order to decide which devices are
>>>>>>> available and which are not inside of a PVH guest.
>>>>>>>
>>>>>>> For example in the FADT "IA-PC Boot Architecture Flags" field for PVH
>>>>>>> we already set "VGA Not Present" and "CMOS RTC Not Present". There
>>>>>>> might be other flags/fields that must be set, but I would like to
>>>>>>> avoid having a CPUID bit or similar saying "PVH", because then Xen
>>>>>>> will be tied to always providing the same set of devices in PVH
>>>>>>> containers.
>>>>>> Why? This would depend on the semantics tied to the flag. It could just
>>>>>> mean "don't assume availability of legacy stuff" (e.g. BIOS calls).
>>>>>>
>>>>>> Linux would have a problem with the ACPI approach as it would try BIOS
>>>>>> calls way before it is initializing its ACPI handling. So in Linux I'd
>>>>>> need another way to tell I'm running in PVH mode, e.g. a "no legacy"
>>>>>> bit in the Xen HVM cpuid leaf.
>>>>> If you are booted from the PVH entry point, there's no BIOS or UEFI
>>>>> (ie: no firmware), if you are booted from the BIOS entry point there's
>>>>> a BIOS and the same applies to UEFI. How does Linux differentiate
>>>>> whether it's booted from BIOS or UEFI?
>>>> They use different entries.
>>> In fact, we had a discussion with Matt Fleming (Linux EFI maintainer) to
>>> see if we can use EFI entry point to also be able to boot PVH guest but
>>> found some issues with that approach, which is why we ended up with a
>>> dedicated PVH entry point.
>>>
>>> I am curious though, Juergen --- what do we need besides zeropage to
>>> allow us to boot PVH from startup_64?
>> Oh, you are right. I managed to get lost in the early boot paths.
>>
>> Only setting up the hyperpage seems to be missing, but this should be
>> doable. And setting xen_pvh, of course.
>
> That last part was actually my question --- do we need to have xen_pvh
> set before we get to xen-specific code for the first time (which I think
> is init_hypervisor_platform()) from startup_64?
>
> Because if we do --- who will set it?
This should be easily testable: Just set an internal indicator in
xen_prepare_pvh() and copy that to xen_pvh in init_hypervisor_platform()
and see whether the kernel still comes up in PVH mode.
The same can be done with setting up the hypercall page.
Just building a kernel trying that...
Juergen
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: Xen PVH support in grub2
2017-11-03 14:59 ` Juergen Gross
2017-11-03 15:10 ` Boris Ostrovsky
@ 2017-11-03 15:10 ` Boris Ostrovsky
1 sibling, 0 replies; 73+ messages in thread
From: Boris Ostrovsky @ 2017-11-03 15:10 UTC (permalink / raw)
To: Juergen Gross, Roger Pau Monné
Cc: The development of GNU GRUB, Daniel Kiper, xen-devel
On 11/03/2017 10:59 AM, Juergen Gross wrote:
> On 03/11/17 15:36, Boris Ostrovsky wrote:
>> On 11/03/2017 10:24 AM, Juergen Gross wrote:
>>> On 03/11/17 15:07, Roger Pau Monné wrote:
>>>> On Fri, Nov 03, 2017 at 01:50:11PM +0100, Juergen Gross wrote:
>>>>> On 03/11/17 13:17, Roger Pau Monné wrote:
>>>>>> On Fri, Nov 03, 2017 at 01:00:46PM +0100, Juergen Gross wrote:
>>>>>>> On 29/09/17 17:51, Roger Pau Monné wrote:
>>>>>>>> On Fri, Sep 29, 2017 at 03:33:58PM +0000, Juergen Gross wrote:
>>>>>>>>> On 29/09/17 17:24, Roger Pau Monné wrote:
>>>>>>>>>> On Fri, Sep 29, 2017 at 02:46:53PM +0000, Juergen Gross wrote:
>>>>>>>>>> Then, I also wonder whether it would make sense for this grub to load
>>>>>>>>>> the kernel using the PVH entry point or the native entry point. Would
>>>>>>>>>> it be possible to boot a Linux kernel up to the point where cpuid can
>>>>>>>>>> be used inside of a PVH container?
>>>>>>>>> I don't think today's Linux allows that. This has been discussed
>>>>>>>>> very thoroughly at the time Boris added PVH V2 support to the kernel.
>>>>>>>> OK, I'm not going to insist on that, but my plans for FreeBSD is to
>>>>>>>> make the native entry point capable of booting inside of a PVH
>>>>>>>> container up to the point where cpuid (or whatever method) can be used
>>>>>>>> to detect the environment.
>>>>>>> Looking more thoroughly into the Linux boot code I think this could
>>>>>>> work for Linux, too. But only if we can tell PVH from HVM in the guest.
>>>>>>> How would you do that in FreeBSD? Via flags in the boot params? This
>>>>>>> would the have to be done in the boot loader (e.g. grub or OVMF).
>>>>>> My plan was not to differentiate between HVM and PVH, but rather to
>>>>>> make use of the ACPI information in order to decide which devices are
>>>>>> available and which are not inside of a PVH guest.
>>>>>>
>>>>>> For example in the FADT "IA-PC Boot Architecture Flags" field for PVH
>>>>>> we already set "VGA Not Present" and "CMOS RTC Not Present". There
>>>>>> might be other flags/fields that must be set, but I would like to
>>>>>> avoid having a CPUID bit or similar saying "PVH", because then Xen
>>>>>> will be tied to always providing the same set of devices in PVH
>>>>>> containers.
>>>>> Why? This would depend on the semantics tied to the flag. It could just
>>>>> mean "don't assume availability of legacy stuff" (e.g. BIOS calls).
>>>>>
>>>>> Linux would have a problem with the ACPI approach as it would try BIOS
>>>>> calls way before it is initializing its ACPI handling. So in Linux I'd
>>>>> need another way to tell I'm running in PVH mode, e.g. a "no legacy"
>>>>> bit in the Xen HVM cpuid leaf.
>>>> If you are booted from the PVH entry point, there's no BIOS or UEFI
>>>> (ie: no firmware), if you are booted from the BIOS entry point there's
>>>> a BIOS and the same applies to UEFI. How does Linux differentiate
>>>> whether it's booted from BIOS or UEFI?
>>> They use different entries.
>> In fact, we had a discussion with Matt Fleming (Linux EFI maintainer) to
>> see if we can use EFI entry point to also be able to boot PVH guest but
>> found some issues with that approach, which is why we ended up with a
>> dedicated PVH entry point.
>>
>> I am curious though, Juergen --- what do we need besides zeropage to
>> allow us to boot PVH from startup_64?
> Oh, you are right. I managed to get lost in the early boot paths.
>
> Only setting up the hyperpage seems to be missing, but this should be
> doable. And setting xen_pvh, of course.
That last part was actually my question --- do we need to have xen_pvh
set before we get to xen-specific code for the first time (which I think
is init_hypervisor_platform()) from startup_64?
Because if we do --- who will set it?
-boris
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: Xen PVH support in grub2
2017-11-03 14:36 ` [Xen-devel] " Boris Ostrovsky
(?)
(?)
@ 2017-11-03 14:59 ` Juergen Gross
-1 siblings, 0 replies; 73+ messages in thread
From: Juergen Gross @ 2017-11-03 14:59 UTC (permalink / raw)
To: Boris Ostrovsky, Roger Pau Monné
Cc: The development of GNU GRUB, Daniel Kiper, xen-devel
On 03/11/17 15:36, Boris Ostrovsky wrote:
> On 11/03/2017 10:24 AM, Juergen Gross wrote:
>> On 03/11/17 15:07, Roger Pau Monné wrote:
>>> On Fri, Nov 03, 2017 at 01:50:11PM +0100, Juergen Gross wrote:
>>>> On 03/11/17 13:17, Roger Pau Monné wrote:
>>>>> On Fri, Nov 03, 2017 at 01:00:46PM +0100, Juergen Gross wrote:
>>>>>> On 29/09/17 17:51, Roger Pau Monné wrote:
>>>>>>> On Fri, Sep 29, 2017 at 03:33:58PM +0000, Juergen Gross wrote:
>>>>>>>> On 29/09/17 17:24, Roger Pau Monné wrote:
>>>>>>>>> On Fri, Sep 29, 2017 at 02:46:53PM +0000, Juergen Gross wrote:
>>>>>>>>> Then, I also wonder whether it would make sense for this grub to load
>>>>>>>>> the kernel using the PVH entry point or the native entry point. Would
>>>>>>>>> it be possible to boot a Linux kernel up to the point where cpuid can
>>>>>>>>> be used inside of a PVH container?
>>>>>>>> I don't think today's Linux allows that. This has been discussed
>>>>>>>> very thoroughly at the time Boris added PVH V2 support to the kernel.
>>>>>>> OK, I'm not going to insist on that, but my plans for FreeBSD is to
>>>>>>> make the native entry point capable of booting inside of a PVH
>>>>>>> container up to the point where cpuid (or whatever method) can be used
>>>>>>> to detect the environment.
>>>>>> Looking more thoroughly into the Linux boot code I think this could
>>>>>> work for Linux, too. But only if we can tell PVH from HVM in the guest.
>>>>>> How would you do that in FreeBSD? Via flags in the boot params? This
>>>>>> would the have to be done in the boot loader (e.g. grub or OVMF).
>>>>> My plan was not to differentiate between HVM and PVH, but rather to
>>>>> make use of the ACPI information in order to decide which devices are
>>>>> available and which are not inside of a PVH guest.
>>>>>
>>>>> For example in the FADT "IA-PC Boot Architecture Flags" field for PVH
>>>>> we already set "VGA Not Present" and "CMOS RTC Not Present". There
>>>>> might be other flags/fields that must be set, but I would like to
>>>>> avoid having a CPUID bit or similar saying "PVH", because then Xen
>>>>> will be tied to always providing the same set of devices in PVH
>>>>> containers.
>>>> Why? This would depend on the semantics tied to the flag. It could just
>>>> mean "don't assume availability of legacy stuff" (e.g. BIOS calls).
>>>>
>>>> Linux would have a problem with the ACPI approach as it would try BIOS
>>>> calls way before it is initializing its ACPI handling. So in Linux I'd
>>>> need another way to tell I'm running in PVH mode, e.g. a "no legacy"
>>>> bit in the Xen HVM cpuid leaf.
>>> If you are booted from the PVH entry point, there's no BIOS or UEFI
>>> (ie: no firmware), if you are booted from the BIOS entry point there's
>>> a BIOS and the same applies to UEFI. How does Linux differentiate
>>> whether it's booted from BIOS or UEFI?
>> They use different entries.
>
> In fact, we had a discussion with Matt Fleming (Linux EFI maintainer) to
> see if we can use EFI entry point to also be able to boot PVH guest but
> found some issues with that approach, which is why we ended up with a
> dedicated PVH entry point.
>
> I am curious though, Juergen --- what do we need besides zeropage to
> allow us to boot PVH from startup_64?
Oh, you are right. I managed to get lost in the early boot paths.
Only setting up the hyperpage seems to be missing, but this should be
doable. And setting xen_pvh, of course.
Juergen
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 73+ messages in thread