Re: Xen Platform QoS design discussion

[George Dunlap <George.Dunlap@eu.citrix.com>]

On Tue, May 6, 2014 at 11:06 AM, Andrew Cooper
<andrew.cooper3@citrix.com> wrote:
> On 06/05/14 02:40, Xu, Dongxiao wrote:
>>> -----Original Message-----
>>> From: Xu, Dongxiao
>>> Sent: Sunday, May 04, 2014 8:46 AM
>>> To: Jan Beulich
>>> Cc: Andrew Cooper(andrew.cooper3@citrix.com); Ian Campbell;
>>> xen-devel@lists.xen.org
>>> Subject: RE: Xen Platform QoS design discussion
>>>
>>>> -----Original Message-----
>>>> From: Jan Beulich [mailto:JBeulich@suse.com]
>>>> Sent: Friday, May 02, 2014 8:40 PM
>>>> To: Xu, Dongxiao
>>>> Cc: Andrew Cooper(andrew.cooper3@citrix.com); Ian Campbell;
>>>> xen-devel@lists.xen.org
>>>> Subject: RE: Xen Platform QoS design discussion
>>>>
>>>>>>> On 02.05.14 at 14:30, <dongxiao.xu@intel.com> wrote:
>>>>>>  -----Original Message-----
>>>>>> From: Jan Beulich [mailto:JBeulich@suse.com]
>>>>>> Sent: Friday, May 02, 2014 5:24 PM
>>>>>> To: Xu, Dongxiao
>>>>>> Cc: Andrew Cooper(andrew.cooper3@citrix.com); Ian Campbell;
>>>>>> xen-devel@lists.xen.org
>>>>>> Subject: RE: Xen Platform QoS design discussion
>>>>>>
>>>>>>>>> On 01.05.14 at 02:56, <dongxiao.xu@intel.com> wrote:
>>>>>>>> From: Ian Campbell [mailto:Ian.Campbell@citrix.com]
>>>>>>>> Have you asked yourself whether this information even needs to be
>>>>>>>> exposed all the way up to libxl? Who are the expected consumers of this
>>>>>>>> interface? Are they low-level CLI tools (i.e. like xenpm is) or are you
>>>>>>>> expecting toolstacks to plumb this information all the way up to their
>>>>>>>> GUI or CLI (e.g. xl or virsh)?
>>>>>>> The information returned to libxl users is the cache utilization for a
>>>>>>> certain domain in certain socket, and the main consumers are cloud users
>>>>> like
>>>>>>> openstack, etc. Of course, we will also provide an xl command to present
>>>>> such
>>>>>>> information.
>>>>>> To me this doesn't really address the question Ian asked, yet knowing
>>>>>> who's going to be the consumer of the data is also quite relevant for
>>>>>> answering your original question on the method to obtain that data.
>>>>>> Obviously, if the main use of it is per-domain, a domctl would seem like
>>>>>> a suitable approach despite the data being more of sysctl kind. But if
>>>>>> a global view would be more important, that model would seem to make
>>>>>> life needlessly hard for the consumers. In turn, if using a domctl, I tend
>>>>>> to agree that not using shared pages would be preferable; iirc their use
>>>>>> was mainly suggested because of the size of the data.
>>>>> From the discussion with openstack developers, on certain cloud host, all
>>>>> running VM's information (e.g., domain ID) will be stored in a database, and
>>>>> openstack software will use libvirt/XenAPI to query specific domain
>>>>> information. That libvirt/XenAPI API interface basically accepts the domain
>>>>> ID as input parameter and get the domain information, including the platform
>>>>> QoS one.
>>>>>
>>>>> Based on above information, I think we'd better design the QoS hypercall
>>>>> per-domain.
>>>> If you think that this is going to be the only (or at least prevalent)
>>>> usage model, that's probably okay then. But I'm a little puzzled that
>>>> all this effort is just for a single, rather specific consumer. I thought
>>>> that if this is so important to Intel there would be wider interested
>>>> audience.
>> Since there is no further comments, I suppose we all agreed on making the hypercall per-domain and use data copying mechanism between hypervisor and Dom0 tool stack?
>>
>
> No - the onus is very much on you to prove that your API will *not* be
> used in the following way:
>
> every $TIMEPERIOD
>   for each domain
>     for each type of information
>       get-$TYPE-information-for-$DOMAIN
>
>
> Which is the source of my concerns regarding overhead.
>
> As far as I can see, as soon as you provide access to this QoS
> information, higher level toolstacks are going to want all information
> for all domains.  Given your proposed domctl, they will have exactly one
> (bad) way of getting this information.

Is this really going to be that much of a critical path that we need
to even have this discussion?

We have two different hypercalls right now for getting "dominfo": a
domctl and a sysctl.  You use the domctl if you want information about
a single domain, you use sysctl if you want information about all
domains.  The sysctl implementation calls the domctl implementation
internally.

Is there a problem with doing the same thing here?  Or, with starting
with a domctl, and then creating a sysctl if iterating over all
domains (and calling the domctl internally) if we measure the domctl
to be too slow for many callers?

 -George