xenstore domain

xenstore domain

[Juergen Gross]

I'm just playing a little bit with xenstore in an own domain.

I've come across some questions I'd like to have some answers to before
presenting official patches to make this an easy configurable option:

a) As this would need a boot time configuration item I'd like to add
   e.g. /etc/xen/server.conf where such global configuration options
   could be set via directives. Is this generally okay? If yes, which
   format? Easiest way would be entries like
   VAR=value
   which can be either sourced in from shell scripts or can easily be
   parsed in all programming languages. What are the preferences here?

b) Today init-xenstore-domain will require flask to be enabled. An
   alternative would be to add a new domain creation flag to allow the
   domains with that flag set calling xc_domain_getinfo(). Thoughts?

c) In order to be as flexible as possible I think the xenstore domain
   should be allowed to auto-balloon according to it's needs. Is anyone
   already working on ballooning support for mini-os?


Juergen

Re: xenstore domain

[Andrew Cooper]

On 08/12/15 14:44, Juergen Gross wrote:
> I'm just playing a little bit with xenstore in an own domain.
>
> I've come across some questions I'd like to have some answers to before
> presenting official patches to make this an easy configurable option:
>
> a) As this would need a boot time configuration item I'd like to add
>    e.g. /etc/xen/server.conf where such global configuration options
>    could be set via directives. Is this generally okay? If yes, which
>    format? Easiest way would be entries like
>    VAR=value
>    which can be either sourced in from shell scripts or can easily be
>    parsed in all programming languages. What are the preferences here?

Any configuration like this going to be toolstack-specific.  I would
recommend against using a name as generic as that.

/etc/xl.conf already exists, which IMO would be the natural place for
this to live, but it isn't parseable by shell, because of vif notation.

One option might be to alter xl.conf to be compatible with shell
parsing.  It wouldn't be complicated (even in upgrade situations), and
would offer rather more flexibility.

>
> b) Today init-xenstore-domain will require flask to be enabled. An
>    alternative would be to add a new domain creation flag to allow the
>    domains with that flag set calling xc_domain_getinfo(). Thoughts?

Which flag?

>
> c) In order to be as flexible as possible I think the xenstore domain
>    should be allowed to auto-balloon according to it's needs. Is anyone
>    already working on ballooning support for mini-os?

This is the C xenstore?  Have you investigated whether Mirage-based
stubdomains can balloon?

~Andrew

Re: xenstore domain

[Juergen Gross]

On 08/12/15 16:04, Andrew Cooper wrote:
> On 08/12/15 14:44, Juergen Gross wrote:
>> I'm just playing a little bit with xenstore in an own domain.
>>
>> I've come across some questions I'd like to have some answers to before
>> presenting official patches to make this an easy configurable option:
>>
>> a) As this would need a boot time configuration item I'd like to add
>>    e.g. /etc/xen/server.conf where such global configuration options
>>    could be set via directives. Is this generally okay? If yes, which
>>    format? Easiest way would be entries like
>>    VAR=value
>>    which can be either sourced in from shell scripts or can easily be
>>    parsed in all programming languages. What are the preferences here?
> 
> Any configuration like this going to be toolstack-specific.  I would
> recommend against using a name as generic as that.
> 
> /etc/xl.conf already exists, which IMO would be the natural place for
> this to live, but it isn't parseable by shell, because of vif notation.

OTOH that file wouldn't be just for xl. It would be consumed by e.g.
xencommons. Other configuration options I'd plan to add would be
driver domains dedicated to specific interface cards.

> One option might be to alter xl.conf to be compatible with shell
> parsing.  It wouldn't be complicated (even in upgrade situations), and
> would offer rather more flexibility.

Shell parsing could be even handled via a rather simple filter, I guess.

>> b) Today init-xenstore-domain will require flask to be enabled. An
>>    alternative would be to add a new domain creation flag to allow the
>>    domains with that flag set calling xc_domain_getinfo(). Thoughts?
> 
> Which flag?

A new domcr_flag.

>> c) In order to be as flexible as possible I think the xenstore domain
>>    should be allowed to auto-balloon according to it's needs. Is anyone
>>    already working on ballooning support for mini-os?
> 
> This is the C xenstore?  Have you investigated whether Mirage-based
> stubdomains can balloon?

Aren't they based on mini-os, too?


Juergen

Re: xenstore domain

[Andrew Cooper]

On 08/12/15 16:02, Juergen Gross wrote:
> On 08/12/15 16:04, Andrew Cooper wrote:
>> On 08/12/15 14:44, Juergen Gross wrote:
>>> I'm just playing a little bit with xenstore in an own domain.
>>>
>>> I've come across some questions I'd like to have some answers to before
>>> presenting official patches to make this an easy configurable option:
>>>
>>> a) As this would need a boot time configuration item I'd like to add
>>>    e.g. /etc/xen/server.conf where such global configuration options
>>>    could be set via directives. Is this generally okay? If yes, which
>>>    format? Easiest way would be entries like
>>>    VAR=value
>>>    which can be either sourced in from shell scripts or can easily be
>>>    parsed in all programming languages. What are the preferences here?
>> Any configuration like this going to be toolstack-specific.  I would
>> recommend against using a name as generic as that.
>>
>> /etc/xl.conf already exists, which IMO would be the natural place for
>> this to live, but it isn't parseable by shell, because of vif notation.
> OTOH that file wouldn't be just for xl. It would be consumed by e.g.
> xencommons. Other configuration options I'd plan to add would be
> driver domains dedicated to specific interface cards.

It is still logically part of the "xl toolstack infrastructure", but I
accept your point.  The current xl.conf is all about how to create
domains in general, rather than specifically "how I would like my system
configured when starting up".

>
>> One option might be to alter xl.conf to be compatible with shell
>> parsing.  It wouldn't be complicated (even in upgrade situations), and
>> would offer rather more flexibility.
> Shell parsing could be even handled via a rather simple filter, I guess.
>
>>> b) Today init-xenstore-domain will require flask to be enabled. An
>>>    alternative would be to add a new domain creation flag to allow the
>>>    domains with that flag set calling xc_domain_getinfo(). Thoughts?
>> Which flag?
> A new domcr_flag.

Indicating what, precisely?

>
>>> c) In order to be as flexible as possible I think the xenstore domain
>>>    should be allowed to auto-balloon according to it's needs. Is anyone
>>>    already working on ballooning support for mini-os?
>> This is the C xenstore?  Have you investigated whether Mirage-based
>> stubdomains can balloon?
> Aren't they based on mini-os, too?

Heavily modified.  For one, they have fixed suspend/resume.  I don't
know whether they have ballooning, and it isn't completely obvious from
their repo.

Dave: any ideas?

~Andrew

Re: xenstore domain

[Jon Ludlam]

On Tue, Dec 08, 2015 at 04:34:13PM +0000, Andrew Cooper wrote:
> On 08/12/15 16:02, Juergen Gross wrote:
> > On 08/12/15 16:04, Andrew Cooper wrote:
> >> On 08/12/15 14:44, Juergen Gross wrote:
> >>> I'm just playing a little bit with xenstore in an own domain.
...
> >>> c) In order to be as flexible as possible I think the xenstore domain
> >>>    should be allowed to auto-balloon according to it's needs. Is anyone
> >>>    already working on ballooning support for mini-os?
> >> This is the C xenstore?  Have you investigated whether Mirage-based
> >> stubdomains can balloon?
> > Aren't they based on mini-os, too?
> 
> Heavily modified.  For one, they have fixed suspend/resume.  I don't
> know whether they have ballooning, and it isn't completely obvious from
> their repo.
>

Not so very heavily modified any more - the repo being used is here:

https://github.com/talex5/mini-os

and many of the patches are gradually being upstreamed. We've lost the
suspend/resume support, though if I get a day or two at some point I'd
like to forward-port it.

As far as I'm aware, there is no ballooning support in any MirageOS
library.

Jon

> Dave: any ideas?
> 
> ~Andrew
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

Re: xenstore domain

[Juergen Gross]

On 08/12/15 17:34, Andrew Cooper wrote:
> On 08/12/15 16:02, Juergen Gross wrote:
>> On 08/12/15 16:04, Andrew Cooper wrote:
>>> On 08/12/15 14:44, Juergen Gross wrote:
>>>> I'm just playing a little bit with xenstore in an own domain.
>>>>
>>>> I've come across some questions I'd like to have some answers to before
>>>> presenting official patches to make this an easy configurable option:
>>>>
>>>> a) As this would need a boot time configuration item I'd like to add
>>>>    e.g. /etc/xen/server.conf where such global configuration options
>>>>    could be set via directives. Is this generally okay? If yes, which
>>>>    format? Easiest way would be entries like
>>>>    VAR=value
>>>>    which can be either sourced in from shell scripts or can easily be
>>>>    parsed in all programming languages. What are the preferences here?
>>> Any configuration like this going to be toolstack-specific.  I would
>>> recommend against using a name as generic as that.
>>>
>>> /etc/xl.conf already exists, which IMO would be the natural place for
>>> this to live, but it isn't parseable by shell, because of vif notation.
>> OTOH that file wouldn't be just for xl. It would be consumed by e.g.
>> xencommons. Other configuration options I'd plan to add would be
>> driver domains dedicated to specific interface cards.
> 
> It is still logically part of the "xl toolstack infrastructure", but I
> accept your point.  The current xl.conf is all about how to create
> domains in general, rather than specifically "how I would like my system
> configured when starting up".
> 
>>
>>> One option might be to alter xl.conf to be compatible with shell
>>> parsing.  It wouldn't be complicated (even in upgrade situations), and
>>> would offer rather more flexibility.
>> Shell parsing could be even handled via a rather simple filter, I guess.
>>
>>>> b) Today init-xenstore-domain will require flask to be enabled. An
>>>>    alternative would be to add a new domain creation flag to allow the
>>>>    domains with that flag set calling xc_domain_getinfo(). Thoughts?
>>> Which flag?
>> A new domcr_flag.
> 
> Indicating what, precisely?

What I need is the capability to do the XEN_DOMCTL_getdomaininfo
hypercall from the xenstore domain. Question is whether it's better
to tie this special capability to the flag or to name it "is_xenstore".

Thinking more about it, especially regarding a possible enhancement
allowing Dom0 to reboot, I think the is_xenstore variant would be
better. This would allow to look whether a xenstore domain is already
running and connect to that rather than try to start a new one.


Juergen

Re: xenstore domain

[Andrew Cooper]

On 09/12/15 07:34, Juergen Gross wrote:
> On 08/12/15 17:34, Andrew Cooper wrote:
>> On 08/12/15 16:02, Juergen Gross wrote:
>>> On 08/12/15 16:04, Andrew Cooper wrote:
>>>> On 08/12/15 14:44, Juergen Gross wrote:
>>>>> I'm just playing a little bit with xenstore in an own domain.
>>>>>
>>>>> I've come across some questions I'd like to have some answers to before
>>>>> presenting official patches to make this an easy configurable option:
>>>>>
>>>>> a) As this would need a boot time configuration item I'd like to add
>>>>>    e.g. /etc/xen/server.conf where such global configuration options
>>>>>    could be set via directives. Is this generally okay? If yes, which
>>>>>    format? Easiest way would be entries like
>>>>>    VAR=value
>>>>>    which can be either sourced in from shell scripts or can easily be
>>>>>    parsed in all programming languages. What are the preferences here?
>>>> Any configuration like this going to be toolstack-specific.  I would
>>>> recommend against using a name as generic as that.
>>>>
>>>> /etc/xl.conf already exists, which IMO would be the natural place for
>>>> this to live, but it isn't parseable by shell, because of vif notation.
>>> OTOH that file wouldn't be just for xl. It would be consumed by e.g.
>>> xencommons. Other configuration options I'd plan to add would be
>>> driver domains dedicated to specific interface cards.
>> It is still logically part of the "xl toolstack infrastructure", but I
>> accept your point.  The current xl.conf is all about how to create
>> domains in general, rather than specifically "how I would like my system
>> configured when starting up".
>>
>>>> One option might be to alter xl.conf to be compatible with shell
>>>> parsing.  It wouldn't be complicated (even in upgrade situations), and
>>>> would offer rather more flexibility.
>>> Shell parsing could be even handled via a rather simple filter, I guess.
>>>
>>>>> b) Today init-xenstore-domain will require flask to be enabled. An
>>>>>    alternative would be to add a new domain creation flag to allow the
>>>>>    domains with that flag set calling xc_domain_getinfo(). Thoughts?
>>>> Which flag?
>>> A new domcr_flag.
>> Indicating what, precisely?
> What I need is the capability to do the XEN_DOMCTL_getdomaininfo
> hypercall from the xenstore domain. Question is whether it's better
> to tie this special capability to the flag or to name it "is_xenstore".
>
> Thinking more about it, especially regarding a possible enhancement
> allowing Dom0 to reboot, I think the is_xenstore variant would be
> better. This would allow to look whether a xenstore domain is already
> running and connect to that rather than try to start a new one.

If we do indeed want dom0 to be able to reboot, then we definitely do
need some bit of remaining state indicating where xenstore is.

Currently it is the residual knowledge that dom0 has from whether it
started a local daemon, or a stubdomain, but that information would
disappear on a reboot.

In general, I would be against adding extra magic like this to Xen, but
xenstore is already sufficiently magic and critical in a Xen system that
the benefits of this special case probably do outweigh its downsides. 
(I wonder how long it will be until multiple xenstore domains are
suggested in earnest on xen-devel.)

~Andrew

Re: xenstore domain

[Doug Goldstein]

[-- Attachment #1.1: Type: text/plain, Size: 3033 bytes --]

On 12/9/15 1:34 AM, Juergen Gross wrote:
> On 08/12/15 17:34, Andrew Cooper wrote:
>> On 08/12/15 16:02, Juergen Gross wrote:
>>> On 08/12/15 16:04, Andrew Cooper wrote:
>>>> On 08/12/15 14:44, Juergen Gross wrote:
>>>>> I'm just playing a little bit with xenstore in an own domain.
>>>>>
>>>>> I've come across some questions I'd like to have some answers to before
>>>>> presenting official patches to make this an easy configurable option:
>>>>>
>>>>> a) As this would need a boot time configuration item I'd like to add
>>>>>    e.g. /etc/xen/server.conf where such global configuration options
>>>>>    could be set via directives. Is this generally okay? If yes, which
>>>>>    format? Easiest way would be entries like
>>>>>    VAR=value
>>>>>    which can be either sourced in from shell scripts or can easily be
>>>>>    parsed in all programming languages. What are the preferences here?
>>>> Any configuration like this going to be toolstack-specific.  I would
>>>> recommend against using a name as generic as that.
>>>>
>>>> /etc/xl.conf already exists, which IMO would be the natural place for
>>>> this to live, but it isn't parseable by shell, because of vif notation.
>>> OTOH that file wouldn't be just for xl. It would be consumed by e.g.
>>> xencommons. Other configuration options I'd plan to add would be
>>> driver domains dedicated to specific interface cards.
>>
>> It is still logically part of the "xl toolstack infrastructure", but I
>> accept your point.  The current xl.conf is all about how to create
>> domains in general, rather than specifically "how I would like my system
>> configured when starting up".
>>
>>>
>>>> One option might be to alter xl.conf to be compatible with shell
>>>> parsing.  It wouldn't be complicated (even in upgrade situations), and
>>>> would offer rather more flexibility.
>>> Shell parsing could be even handled via a rather simple filter, I guess.
>>>
>>>>> b) Today init-xenstore-domain will require flask to be enabled. An
>>>>>    alternative would be to add a new domain creation flag to allow the
>>>>>    domains with that flag set calling xc_domain_getinfo(). Thoughts?
>>>> Which flag?
>>> A new domcr_flag.
>>
>> Indicating what, precisely?
> 
> What I need is the capability to do the XEN_DOMCTL_getdomaininfo
> hypercall from the xenstore domain. Question is whether it's better
> to tie this special capability to the flag or to name it "is_xenstore".
> 
> Thinking more about it, especially regarding a possible enhancement
> allowing Dom0 to reboot, I think the is_xenstore variant would be
> better. This would allow to look whether a xenstore domain is already
> running and connect to that rather than try to start a new one.
> 
> 
> Juergen

How would either of these relate to /proc/xen/capabilities and/or
/sys/hypervisor/properties/capabilities?

A number of distros use the former to decide when to start up xenstore
(in fact the in tree scripts do as well).

-- 
Doug Goldstein


[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 959 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: xenstore domain

[Juergen Gross]

On 10/12/15 22:13, Doug Goldstein wrote:
> On 12/9/15 1:34 AM, Juergen Gross wrote:
>> On 08/12/15 17:34, Andrew Cooper wrote:
>>> On 08/12/15 16:02, Juergen Gross wrote:
>>>> On 08/12/15 16:04, Andrew Cooper wrote:
>>>>> On 08/12/15 14:44, Juergen Gross wrote:
>>>>>> I'm just playing a little bit with xenstore in an own domain.
>>>>>>
>>>>>> I've come across some questions I'd like to have some answers to before
>>>>>> presenting official patches to make this an easy configurable option:
>>>>>>
>>>>>> a) As this would need a boot time configuration item I'd like to add
>>>>>>    e.g. /etc/xen/server.conf where such global configuration options
>>>>>>    could be set via directives. Is this generally okay? If yes, which
>>>>>>    format? Easiest way would be entries like
>>>>>>    VAR=value
>>>>>>    which can be either sourced in from shell scripts or can easily be
>>>>>>    parsed in all programming languages. What are the preferences here?
>>>>> Any configuration like this going to be toolstack-specific.  I would
>>>>> recommend against using a name as generic as that.
>>>>>
>>>>> /etc/xl.conf already exists, which IMO would be the natural place for
>>>>> this to live, but it isn't parseable by shell, because of vif notation.
>>>> OTOH that file wouldn't be just for xl. It would be consumed by e.g.
>>>> xencommons. Other configuration options I'd plan to add would be
>>>> driver domains dedicated to specific interface cards.
>>>
>>> It is still logically part of the "xl toolstack infrastructure", but I
>>> accept your point.  The current xl.conf is all about how to create
>>> domains in general, rather than specifically "how I would like my system
>>> configured when starting up".
>>>
>>>>
>>>>> One option might be to alter xl.conf to be compatible with shell
>>>>> parsing.  It wouldn't be complicated (even in upgrade situations), and
>>>>> would offer rather more flexibility.
>>>> Shell parsing could be even handled via a rather simple filter, I guess.
>>>>
>>>>>> b) Today init-xenstore-domain will require flask to be enabled. An
>>>>>>    alternative would be to add a new domain creation flag to allow the
>>>>>>    domains with that flag set calling xc_domain_getinfo(). Thoughts?
>>>>> Which flag?
>>>> A new domcr_flag.
>>>
>>> Indicating what, precisely?
>>
>> What I need is the capability to do the XEN_DOMCTL_getdomaininfo
>> hypercall from the xenstore domain. Question is whether it's better
>> to tie this special capability to the flag or to name it "is_xenstore".
>>
>> Thinking more about it, especially regarding a possible enhancement
>> allowing Dom0 to reboot, I think the is_xenstore variant would be
>> better. This would allow to look whether a xenstore domain is already
>> running and connect to that rather than try to start a new one.
>>
>>
>> Juergen
> 
> How would either of these relate to /proc/xen/capabilities and/or
> /sys/hypervisor/properties/capabilities?

Not at all. Both files are dom0 products. And xenstore domain is not
a hypervisor capability, but just a configuration variant of the
system.

> A number of distros use the former to decide when to start up xenstore
> (in fact the in tree scripts do as well).

This is subject to dom0 configuration. In case it is configured to use
a xenstore domain it has to check whether this domain is already
running. The decision to start a new domain is depending on the result
of this test.


Juergen

Re: xenstore domain

[Juergen Gross]

On 10/12/15 21:24, Andrew Cooper wrote:
> On 09/12/15 07:34, Juergen Gross wrote:
>> On 08/12/15 17:34, Andrew Cooper wrote:
>>> On 08/12/15 16:02, Juergen Gross wrote:
>>>> On 08/12/15 16:04, Andrew Cooper wrote:
>>>>> On 08/12/15 14:44, Juergen Gross wrote:
>>>>>> I'm just playing a little bit with xenstore in an own domain.
>>>>>>
>>>>>> I've come across some questions I'd like to have some answers to before
>>>>>> presenting official patches to make this an easy configurable option:
>>>>>>
>>>>>> a) As this would need a boot time configuration item I'd like to add
>>>>>>    e.g. /etc/xen/server.conf where such global configuration options
>>>>>>    could be set via directives. Is this generally okay? If yes, which
>>>>>>    format? Easiest way would be entries like
>>>>>>    VAR=value
>>>>>>    which can be either sourced in from shell scripts or can easily be
>>>>>>    parsed in all programming languages. What are the preferences here?
>>>>> Any configuration like this going to be toolstack-specific.  I would
>>>>> recommend against using a name as generic as that.
>>>>>
>>>>> /etc/xl.conf already exists, which IMO would be the natural place for
>>>>> this to live, but it isn't parseable by shell, because of vif notation.
>>>> OTOH that file wouldn't be just for xl. It would be consumed by e.g.
>>>> xencommons. Other configuration options I'd plan to add would be
>>>> driver domains dedicated to specific interface cards.
>>> It is still logically part of the "xl toolstack infrastructure", but I
>>> accept your point.  The current xl.conf is all about how to create
>>> domains in general, rather than specifically "how I would like my system
>>> configured when starting up".
>>>
>>>>> One option might be to alter xl.conf to be compatible with shell
>>>>> parsing.  It wouldn't be complicated (even in upgrade situations), and
>>>>> would offer rather more flexibility.
>>>> Shell parsing could be even handled via a rather simple filter, I guess.
>>>>
>>>>>> b) Today init-xenstore-domain will require flask to be enabled. An
>>>>>>    alternative would be to add a new domain creation flag to allow the
>>>>>>    domains with that flag set calling xc_domain_getinfo(). Thoughts?
>>>>> Which flag?
>>>> A new domcr_flag.
>>> Indicating what, precisely?
>> What I need is the capability to do the XEN_DOMCTL_getdomaininfo
>> hypercall from the xenstore domain. Question is whether it's better
>> to tie this special capability to the flag or to name it "is_xenstore".
>>
>> Thinking more about it, especially regarding a possible enhancement
>> allowing Dom0 to reboot, I think the is_xenstore variant would be
>> better. This would allow to look whether a xenstore domain is already
>> running and connect to that rather than try to start a new one.
> 
> If we do indeed want dom0 to be able to reboot, then we definitely do
> need some bit of remaining state indicating where xenstore is.

Indeed.

> Currently it is the residual knowledge that dom0 has from whether it
> started a local daemon, or a stubdomain, but that information would
> disappear on a reboot.
> 
> In general, I would be against adding extra magic like this to Xen, but
> xenstore is already sufficiently magic and critical in a Xen system that
> the benefits of this special case probably do outweigh its downsides. 
> (I wonder how long it will be until multiple xenstore domains are
> suggested in earnest on xen-devel.)

This again is subject to the capabilities of the xenstore domain (can it
handle multiple instances) and of dom0. I didn't plan to limit the
is_xenstore flag to just one domain. My current patches wouldn't start
another xenstore domain in case there is already one running, but this
is a local decision of init-xenstore-domain.


Juergen