Runstate hypercall and Linux KPTI issues

Runstate hypercall and Linux KPTI issues

[Bertrand Marquis]

Hi,

Following my patch[1] to map the guest runstate in Xen during the hypercall directly
instead of doing the conversion from virtual to physical when updating the runstate
content during a context switch a global discussion started[2][3].

To resume the discussion the problem is the following: A guest registers a memory
area for xen to put on and maintain the runstate information. This is done currently
using a guest virtual address which is converted by Xen during context switches.
When KPTI is used and a context switch occurs while linux is running in user space
the area is not mapped and the information is not updated. This results in invalid
runstate information but also in some annoying warning coming up in Xen console
on arm.

After the discussion by mail and the last community call it was proposed to change
the way to go and instead of trying to fix the problem in the existing hypercall, to
introduce a new hypercall taking as parameter a guest physical address for the
runstate area instead of a virtual address in the current hypercall.

This means:
- add a new hypercall to Xen
- add support for this new hypercall in Linux and use it if Xen supports it
- keep existing hypercall with its limitation (for older guests)
- keep support for both behaviour during the context switch

Some open questions:
- should we allow to register an area using both hypercalls or should it be exclusive ?
- should we backport the support for this hypercall in older kernel releases ?
- other ?

Please tell me if you agree or not before I start to plan how this can be implemented :-)

Regards
Bertrand

[1] https://lists.xenproject.org/archives/html/xen-devel/2020-07/msg01541.html
[2] https://lists.xenproject.org/archives/html/xen-devel/2020-07/msg01461.html
[3] https://lists.xenproject.org/archives/html/xen-devel/2020-06/msg00810.html





IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.

Re: Runstate hypercall and Linux KPTI issues

[Jan Beulich]

On 10.09.2020 15:46, Bertrand Marquis wrote:
> Some open questions:
> - should we allow to register an area using both hypercalls or should it be exclusive ?

I thought it was already clarified that to a certain degree both need
to remain usable at least in sequence, to allow transitioning control
between entirely independent entities (bootloader -> kernel -> dump-
kernel, for example).

> - should we backport the support for this hypercall in older kernel releases ?

It's a bug fix to KPTI, and as such ought to be at least eligible for
considering doing so?

Jan

Re: Runstate hypercall and Linux KPTI issues

[Bertrand Marquis]

> On 10 Sep 2020, at 14:56, Jan Beulich <jbeulich@suse.com> wrote:
>
> On 10.09.2020 15:46, Bertrand Marquis wrote:
>> Some open questions:
>> - should we allow to register an area using both hypercalls or should it be exclusive ?
>
> I thought it was already clarified that to a certain degree both need
> to remain usable at least in sequence, to allow transitioning control
> between entirely independent entities (bootloader -> kernel -> dump-
> kernel, for example).

Sorry my wording was not clear here

Should we allow to register 2 areas at the same time using both hypercalls (one with
virtual address and one with physical address) or should they be exclusive ie one or
the other but not both at the same time

>
>> - should we backport the support for this hypercall in older kernel releases ?
>
> It's a bug fix to KPTI, and as such ought to be at least eligible for
> considering doing so?

That will mean also backport in Linux. What should be the scope ?

Bertrand

>
> Jan

IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.

Re: Runstate hypercall and Linux KPTI issues

[Bertrand Marquis]

> On 10 Sep 2020, at 15:00, Bertrand Marquis <Bertrand.Marquis@arm.com> wrote:
> 
> 
> 
>> On 10 Sep 2020, at 14:56, Jan Beulich <jbeulich@suse.com> wrote:
>> 
>> On 10.09.2020 15:46, Bertrand Marquis wrote:
>>> Some open questions:
>>> - should we allow to register an area using both hypercalls or should it be exclusive ?
>> 
>> I thought it was already clarified that to a certain degree both need
>> to remain usable at least in sequence, to allow transitioning control
>> between entirely independent entities (bootloader -> kernel -> dump-
>> kernel, for example).
> 
> Sorry my wording was not clear here
> 
> Should we allow to register 2 areas at the same time using both hypercalls (one with
> virtual address and one with physical address) or should they be exclusive ie one or
> the other but not both at the same time
> 
>> 
>>> - should we backport the support for this hypercall in older kernel releases ?
>> 
>> It's a bug fix to KPTI, and as such ought to be at least eligible for
>> considering doing so?
> 
> That will mean also backport in Linux. What should be the scope ?
> 
> Bertrand
> 
>> 
>> Jan
> 
> IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.

Sorry the disclaimer should not appear anymore, no idea what is going wrong on our side here

Bertrand

Re: Runstate hypercall and Linux KPTI issues

[Jan Beulich]

On 10.09.2020 16:00, Bertrand Marquis wrote:
>> On 10 Sep 2020, at 14:56, Jan Beulich <jbeulich@suse.com> wrote:
>> On 10.09.2020 15:46, Bertrand Marquis wrote:
>>> Some open questions:
>>> - should we allow to register an area using both hypercalls or should it be exclusive ?
>>
>> I thought it was already clarified that to a certain degree both need
>> to remain usable at least in sequence, to allow transitioning control
>> between entirely independent entities (bootloader -> kernel -> dump-
>> kernel, for example).
> 
> Sorry my wording was not clear here
> 
> Should we allow to register 2 areas at the same time using both hypercalls (one with
> virtual address and one with physical address) or should they be exclusive ie one or
> the other but not both at the same time

Ah, okay. Just one area at a time, I would say.

>>> - should we backport the support for this hypercall in older kernel releases ?
>>
>> It's a bug fix to KPTI, and as such ought to be at least eligible for
>> considering doing so?
> 
> That will mean also backport in Linux. What should be the scope ?

All longterm and stable trees which are affected, as I think is usual.

Jan

Re: Runstate hypercall and Linux KPTI issues

[Stefano Stabellini]

On Thu, 10 Sep 2020, Jan Beulich wrote:
> >>> - should we backport the support for this hypercall in older kernel releases ?
> >>
> >> It's a bug fix to KPTI, and as such ought to be at least eligible for
> >> considering doing so?
> > 
> > That will mean also backport in Linux. What should be the scope ?
> 
> All longterm and stable trees which are affected, as I think is usual.

From a Linux perspective, we should fix this as soon as possible: we
should backport a patch to make the usage of the runstate hypercall
conditional on KPTI being disabled on ARM.

Then, when available in Xen, we should backport the usage of the new
hypercall with a check to detect if it is available -- do not assume it
is available, users might not update Xen, but might update Linux.

We have to do this in two stages for a couple of reasons. It is best not
to wait for Xen-side changes to fix Linux any longer. And also, a Linux
fix is best implemented independently anyway: Linux should benefit from
the Xen improvements when available but not rely on their presence to
work.

Re: Runstate hypercall and Linux KPTI issues

[Bertrand Marquis]

Hi,

Sorry for the delay.

> On 11 Sep 2020, at 01:33, Stefano Stabellini <sstabellini@kernel.org> wrote:
>
> On Thu, 10 Sep 2020, Jan Beulich wrote:
>>>>> - should we backport the support for this hypercall in older kernel releases ?
>>>>
>>>> It's a bug fix to KPTI, and as such ought to be at least eligible for
>>>> considering doing so?
>>>
>>> That will mean also backport in Linux. What should be the scope ?
>>
>> All longterm and stable trees which are affected, as I think is usual.
>
> From a Linux perspective, we should fix this as soon as possible: we
> should backport a patch to make the usage of the runstate hypercall
> conditional on KPTI being disabled on ARM.
>
> Then, when available in Xen, we should backport the usage of the new
> hypercall with a check to detect if it is available -- do not assume it
> is available, users might not update Xen, but might update Linux.
>
> We have to do this in two stages for a couple of reasons. It is best not
> to wait for Xen-side changes to fix Linux any longer. And also, a Linux
> fix is best implemented independently anyway: Linux should benefit from
> the Xen improvements when available but not rely on their presence to
> work.

I fully agree with that and this should solve the short term issue.

In the long term the new hypercall seems to be the only solution but
it is not a small change and I will not have time to work on this in the
near future.

Cheers
Bertrand




IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.

Re: Runstate hypercall and Linux KPTI issues

[Bertrand Marquis]

> On 24 Sep 2020, at 18:25, Bertrand Marquis <Bertrand.Marquis@arm.com> wrote:
> 
> Hi,
> 
> Sorry for the delay.
> 
>> On 11 Sep 2020, at 01:33, Stefano Stabellini <sstabellini@kernel.org> wrote:
>> 
>> On Thu, 10 Sep 2020, Jan Beulich wrote:
>>>>>> - should we backport the support for this hypercall in older kernel releases ?
>>>>> 
>>>>> It's a bug fix to KPTI, and as such ought to be at least eligible for
>>>>> considering doing so?
>>>> 
>>>> That will mean also backport in Linux. What should be the scope ?
>>> 
>>> All longterm and stable trees which are affected, as I think is usual.
>> 
>> From a Linux perspective, we should fix this as soon as possible: we
>> should backport a patch to make the usage of the runstate hypercall
>> conditional on KPTI being disabled on ARM.
>> 
>> Then, when available in Xen, we should backport the usage of the new
>> hypercall with a check to detect if it is available -- do not assume it
>> is available, users might not update Xen, but might update Linux.
>> 
>> We have to do this in two stages for a couple of reasons. It is best not
>> to wait for Xen-side changes to fix Linux any longer. And also, a Linux
>> fix is best implemented independently anyway: Linux should benefit from
>> the Xen improvements when available but not rely on their presence to
>> work.
> 
> I fully agree with that and this should solve the short term issue.
> 
> In the long term the new hypercall seems to be the only solution but
> it is not a small change and I will not have time to work on this in the
> near future.
> 
> Cheers
> Bertrand
> 

Sorry for the notice, it seems our mail server is adding it randomly…

Bertrand

> 
> 
> 
> IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
>