per-domain configuration and inappropriate use of globals

per-domain configuration and inappropriate use of globals

[Andrew Cooper]

Hello,

I noticed this most recently in the AVIC series from Janakarajan.  The
global svm_avic boolean was left off-by-default because it doesn't work
with nested virt yet.  The code in question was actually inherited from
the VT-x side, and the general problem is systemic with how Xen has been
developed in the past.

Nested virt is the easiest way to spot why this is an issue.  How to
correctly inject an interrupt into a VM depends on the current
configuration in the VMCB/VMCS, and in the case of nested virt, this is
chosen (at least in part) by the L1 hypervisor.  Therefore, a global
boolean isn't correct, because it cannot account for the fact that two
VMCBs/VMCSs might be configured differently.

Other settings I've noticed recently are hap/iommu PT sharing, and
unrestricted_guest, both of which would be more convenient for
development if they were configurable per domain rather than requiring a
host reboot to change.

In practice, having fine grain control of all the features like would be
excellent for testing purposes, because it allows you to boot two
otherwise-identical VMs with one configuration difference between them.

In the spirit of the already in progress domaincreate work, options like
these should be selectable at domain creation time, and immutable
thereafter.

That said, there is a plethora of tweakables, and I'm not sure how best
to expose them.  While most (all?) of these options are inherently
supported (as playing with them simulates what Xen would chose on
different hardware), I expect there will be ample opportunity for people
to break their systems if they tweak too much.

Is there liable to be any provision in xl/libxl to have "unstable"
configuration, which is easily identified as "may stop working / cease
to exist / become invalid at any point in the future?"

Alternatively, are there any other suggestions for alternative mechanisms?

Thanks,

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: per-domain configuration and inappropriate use of globals

[Juergen Gross]

On 19/10/2018 18:57, Andrew Cooper wrote:
> In practice, having fine grain control of all the features like would be
> excellent for testing purposes, because it allows you to boot two
> otherwise-identical VMs with one configuration difference between them.
> 
> In the spirit of the already in progress domaincreate work, options like
> these should be selectable at domain creation time, and immutable
> thereafter.
> 
> That said, there is a plethora of tweakables, and I'm not sure how best
> to expose them.  While most (all?) of these options are inherently
> supported (as playing with them simulates what Xen would chose on
> different hardware), I expect there will be ample opportunity for people
> to break their systems if they tweak too much.
> 
> Is there liable to be any provision in xl/libxl to have "unstable"
> configuration, which is easily identified as "may stop working / cease
> to exist / become invalid at any point in the future?"
> 
> Alternatively, are there any other suggestions for alternative mechanisms?

Per-domain parameters like in my series? You could guard the "dangerous"
ones by a global parameter (boot-time or run-time settable).


Juergen

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: per-domain configuration and inappropriate use of globals

[Andrew Cooper]

On 19/10/18 19:22, Juergen Gross wrote:
> On 19/10/2018 18:57, Andrew Cooper wrote:
>> In practice, having fine grain control of all the features like would be
>> excellent for testing purposes, because it allows you to boot two
>> otherwise-identical VMs with one configuration difference between them.
>>
>> In the spirit of the already in progress domaincreate work, options like
>> these should be selectable at domain creation time, and immutable
>> thereafter.
>>
>> That said, there is a plethora of tweakables, and I'm not sure how best
>> to expose them.  While most (all?) of these options are inherently
>> supported (as playing with them simulates what Xen would chose on
>> different hardware), I expect there will be ample opportunity for people
>> to break their systems if they tweak too much.
>>
>> Is there liable to be any provision in xl/libxl to have "unstable"
>> configuration, which is easily identified as "may stop working / cease
>> to exist / become invalid at any point in the future?"
>>
>> Alternatively, are there any other suggestions for alternative mechanisms?
> Per-domain parameters like in my series? You could guard the "dangerous"
> ones by a global parameter (boot-time or run-time settable).

I was hoping to separate the discussion of what information should be
configurable, from the mechanism we used to provide said information.

Using a text-based mechanism suffers from the same stable/unstable
issues as xl.cfg, so the same concern applies there.

(Also, I have a separate uneasy feeling about having yet more string
parsing in the hypervisor, but that is definitely unrelated).

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: per-domain configuration and inappropriate use of globals

[Jan Beulich]

>>> On 19.10.18 at 18:57, <andrew.cooper3@citrix.com> wrote:
> Hello,
> 
> I noticed this most recently in the AVIC series from Janakarajan.  The
> global svm_avic boolean was left off-by-default because it doesn't work
> with nested virt yet.  The code in question was actually inherited from
> the VT-x side, and the general problem is systemic with how Xen has been
> developed in the past.
> 
> Nested virt is the easiest way to spot why this is an issue.  How to
> correctly inject an interrupt into a VM depends on the current
> configuration in the VMCB/VMCS, and in the case of nested virt, this is
> chosen (at least in part) by the L1 hypervisor.  Therefore, a global
> boolean isn't correct, because it cannot account for the fact that two
> VMCBs/VMCSs might be configured differently.
> 
> Other settings I've noticed recently are hap/iommu PT sharing, and
> unrestricted_guest, both of which would be more convenient for
> development if they were configurable per domain rather than requiring a
> host reboot to change.
> 
> In practice, having fine grain control of all the features like would be
> excellent for testing purposes, because it allows you to boot two
> otherwise-identical VMs with one configuration difference between them.
> 
> In the spirit of the already in progress domaincreate work, options like
> these should be selectable at domain creation time, and immutable
> thereafter.
> 
> That said, there is a plethora of tweakables, and I'm not sure how best
> to expose them.  While most (all?) of these options are inherently
> supported (as playing with them simulates what Xen would chose on
> different hardware), I expect there will be ample opportunity for people
> to break their systems if they tweak too much.

Wouldn't this be the case with globals as well? I'd prefer such choice
to be done through generic or arch-specific flags to domain creation.

Jan



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: per-domain configuration and inappropriate use of globals

[Wei Liu]

On Mon, Oct 22, 2018 at 02:29:40PM +0100, Andrew Cooper wrote:
> On 19/10/18 19:22, Juergen Gross wrote:
> > On 19/10/2018 18:57, Andrew Cooper wrote:
> >> In practice, having fine grain control of all the features like would be
> >> excellent for testing purposes, because it allows you to boot two
> >> otherwise-identical VMs with one configuration difference between them.
> >>
> >> In the spirit of the already in progress domaincreate work, options like
> >> these should be selectable at domain creation time, and immutable
> >> thereafter.
> >>
> >> That said, there is a plethora of tweakables, and I'm not sure how best
> >> to expose them.  While most (all?) of these options are inherently
> >> supported (as playing with them simulates what Xen would chose on
> >> different hardware), I expect there will be ample opportunity for people
> >> to break their systems if they tweak too much.
> >>
> >> Is there liable to be any provision in xl/libxl to have "unstable"
> >> configuration, which is easily identified as "may stop working / cease
> >> to exist / become invalid at any point in the future?"
> >>
> >> Alternatively, are there any other suggestions for alternative mechanisms?
> > Per-domain parameters like in my series? You could guard the "dangerous"
> > ones by a global parameter (boot-time or run-time settable).
> 
> I was hoping to separate the discussion of what information should be
> configurable, from the mechanism we used to provide said information.

How do you plan to express this information in SUPPORT.md?

> 
> Using a text-based mechanism suffers from the same stable/unstable
> issues as xl.cfg, so the same concern applies there.
> 

But that is a get-out-of-jail-free card for xl / libxl because they
wouldn't need to care what is supplied. :p

If the way you want this to work is inherently unstable, I think it'd be
better to leave xl / libxl out of the picture from the beginning.

Wei.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel