Re: Xen Platform QoS design discussion

[George Dunlap <george.dunlap@eu.citrix.com>]

On 05/19/2014 12:45 PM, Jan Beulich wrote:
>>>> On 19.05.14 at 13:28, <George.Dunlap@eu.citrix.com> wrote:
>> But in reality, all we need the daemon for is a place to store the
>> information to query.  The idea we came up with was to allocate memory
>> *inside the hypervisor* to store the information.  The idea is that
>> we'd have a sysctl to prompt Xen to *collect* the data into some
>> memory buffers inside of Xen, and then a domctl that would allow you
>> query the data on a per-domain basis.
>>
>> That should be a good balance -- it's not quite as good as having as
>> separate daemon, but it's a pretty good compromise.
> Which all leaves aside the suggested alternative of making available
> a couple of simple operations allowing an eventual daemon to do the
> MSR accesses without the hypervisor being concerned about where
> to store the data and how to make it accessible to the consumer.

 From a libxl perspective, if we provide "libxl_qos_refresh()" (or 
"libxl_qos_freshness_set()") and "libxl_qos_domain_query()" (or 
something like it), it doesn't matter whether it's backed by memory 
stored in Xen via hypercall or by a daemon.

What I was actually envisioning was an option to either query them by a 
domctl hypercall, or by having a daemon map the pages and read them 
directly.  That way we have the daemon available for those who want it 
(say, maybe xapi, or a future libxl daemon / stat collector), but we can 
get a basic level implemented right now without a terrible amount of 
architectural work.

>> There are a couple of options regarding collecting the data.  One is
>> to simply require the caller to do a "poll" sysctl every time they
>> want to refresh the data.  Another possibility would be to have a
>> sysctl "freshness" knob: you could say, "Please make sure the data is
>> no more than 1000ms old"; Xen could then automatically do a refresh
>> when necessary.
>>
>> The advantage of the "poll" method is that you could get a consistent
>> snapshot across all domains; but you'd have to add in code to do the
>> refresh.  (An xl command querying an individual domain would
>> undoubtedly end up calling the poll on each execution, for instance.)
>>
>> An advantage of the "freshness" knob, on the other hand, is that you
>> automatically get coalescing without having to do anything special
>> with the interface.
> With the clear disadvantage of potentially doing work the results of
> which is never going to be looked at by anyone.

Jan, when you make a criticism it needs to be clear what alternate you 
are suggesting.

AFAICT, regarding "collection" of the data, we have exactly three options:
A. Implement a "collect for all domains" option (with an additional 
"query data for a single domain" mechanism; either by daemon or hypercall).
B. Implement a "collect information for a single domain at a time" option
C. Implement both options.

"Doing work that is never looked at by anyone" will always be a 
potential problem if we choose A, whether we use a daemon, or use the 
polling method, or use the automatic "freshness" knob.  The only way to 
avoid that is to do B or C.

We've already said that we expect the common case we expect is for a 
toolstack to want to query all domains anyway.  If we think that's true, 
"make the common case fast and the uncommon case correct" would dictate 
against B.

So are you suggesting B (disputing the expected use case)?  Or are you 
suggesting C?  Or are you just finding fault without thinking things 
through?

  -George