From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Xu, Dongxiao" Subject: Re: Xen Platform QoS design discussion Date: Thu, 22 May 2014 08:19:10 +0000 Message-ID: <40776A41FC278F40B59438AD47D147A911A1AAEC@SHSMSX104.ccr.corp.intel.com> References: <40776A41FC278F40B59438AD47D147A9119F3FEA@SHSMSX104.ccr.corp.intel.com> <1398877340.5166.13.camel@kazak.uk.xensource.com> <40776A41FC278F40B59438AD47D147A9119F42A2@SHSMSX104.ccr.corp.intel.com> <5363804B020000780000E604@mail.emea.novell.com> <40776A41FC278F40B59438AD47D147A9119F4EF4@SHSMSX104.ccr.corp.intel.com> <5363AE54020000780000E7A2@mail.emea.novell.com> <40776A41FC278F40B59438AD47D147A9119FE6BB@SHSMSX104.ccr.corp.intel.com> <5368B418.9000307@citrix.com> <536AA342.8030003@citrix.com> <40776A41FC278F40B59438AD47D147A911A00A4C@SHSMSX104.ccr.corp.intel.com> <536B69AB.7010005@citrix.com> <40776A41FC278F40B59438AD47D147A911A150FC@SHSMSX104.ccr.corp.intel.com> <537A0B17020000780001390F@mail.emea.novell.com> <5379F576.4050108@eu.citrix.com> <537A18260200007800013A06@mail.emea.novell.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <537A18260200007800013A06@mail.emea.novell.com> Content-Language: en-US List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Jan Beulich , George Dunlap Cc: Andrew Cooper , Ian Campbell , "xen-devel@lists.xen.org" List-Id: xen-devel@lists.xenproject.org > -----Original Message----- > From: xen-devel-bounces@lists.xen.org > [mailto:xen-devel-bounces@lists.xen.org] On Behalf Of Jan Beulich > Sent: Monday, May 19, 2014 8:42 PM > To: George Dunlap > Cc: Andrew Cooper; Xu, Dongxiao; Ian Campbell; xen-devel@lists.xen.org > Subject: Re: [Xen-devel] Xen Platform QoS design discussion > > >>> On 19.05.14 at 14:13, wrote: > > On 05/19/2014 12:45 PM, Jan Beulich wrote: > >>>>> On 19.05.14 at 13:28, wrote: > >>> But in reality, all we need the daemon for is a place to store the > >>> information to query. The idea we came up with was to allocate memory > >>> *inside the hypervisor* to store the information. The idea is that > >>> we'd have a sysctl to prompt Xen to *collect* the data into some > >>> memory buffers inside of Xen, and then a domctl that would allow you > >>> query the data on a per-domain basis. > >>> > >>> That should be a good balance -- it's not quite as good as having as > >>> separate daemon, but it's a pretty good compromise. > >> Which all leaves aside the suggested alternative of making available > >> a couple of simple operations allowing an eventual daemon to do the > >> MSR accesses without the hypervisor being concerned about where > >> to store the data and how to make it accessible to the consumer. > > > > From a libxl perspective, if we provide "libxl_qos_refresh()" (or > > "libxl_qos_freshness_set()") and "libxl_qos_domain_query()" (or > > something like it), it doesn't matter whether it's backed by memory > > stored in Xen via hypercall or by a daemon. > > > > What I was actually envisioning was an option to either query them by a > > domctl hypercall, or by having a daemon map the pages and read them > > directly. That way we have the daemon available for those who want it > > (say, maybe xapi, or a future libxl daemon / stat collector), but we can > > get a basic level implemented right now without a terrible amount of > > architectural work. > > But that's all centric towards the daemon concept (if we consider > storing the data inn hypervisor memory also being some kind of a > daemon). Whereas the simple helpers I'm suggesting wouldn't > necessarily require a daemon to be written at all - a query > operation for a domain would then simply be broken down at the > tools level to a number of MSR writes/reads. > > >>> There are a couple of options regarding collecting the data. One is > >>> to simply require the caller to do a "poll" sysctl every time they > >>> want to refresh the data. Another possibility would be to have a > >>> sysctl "freshness" knob: you could say, "Please make sure the data is > >>> no more than 1000ms old"; Xen could then automatically do a refresh > >>> when necessary. > >>> > >>> The advantage of the "poll" method is that you could get a consistent > >>> snapshot across all domains; but you'd have to add in code to do the > >>> refresh. (An xl command querying an individual domain would > >>> undoubtedly end up calling the poll on each execution, for instance.) > >>> > >>> An advantage of the "freshness" knob, on the other hand, is that you > >>> automatically get coalescing without having to do anything special > >>> with the interface. > >> With the clear disadvantage of potentially doing work the results of > >> which is never going to be looked at by anyone. > > > > Jan, when you make a criticism it needs to be clear what alternate you > > are suggesting. > > With there only having been given two options, it seemed clear that > by seeing an obvious downside for one I would mean to other to be > preferable. Of course you're right in saying (further down) that the > risk of obtaining data that no-one is interested in is always there, > just that when someone says "poll" I'd imply (s)he's interested in the > data, as opposed to doing the collect periodically. > > > AFAICT, regarding "collection" of the data, we have exactly three options: > > A. Implement a "collect for all domains" option (with an additional > > "query data for a single domain" mechanism; either by daemon or hypercall). > > B. Implement a "collect information for a single domain at a time" option > > C. Implement both options. > > > > "Doing work that is never looked at by anyone" will always be a > > potential problem if we choose A, whether we use a daemon, or use the > > polling method, or use the automatic "freshness" knob. The only way to > > avoid that is to do B or C. > > > > We've already said that we expect the common case we expect is for a > > toolstack to want to query all domains anyway. If we think that's true, > > "make the common case fast and the uncommon case correct" would dictate > > against B. > > > > So are you suggesting B (disputing the expected use case)? Or are you > > suggesting C? Or are you just finding fault without thinking things > > through? > > I'm certainly putting under question whether the supposed use case > indeed is the common one, and I do that no matter which model > someone claims is going to be the "one". I simply see neither backed > by any sufficiently hard data. > > And without seeing the need for any advanced access mechanism, > I'm continuing to try to promote D - implement simple, policy free > (platform or sysctl) hypercalls providing MSR access to the tool stack > (along the lines of the msr.ko Linux kernel driver). Do you mean some hypercall implementation like following: In this case, Dom0 toolstack actually queries the real physical CPU MSRs. struct xen_sysctl_accessmsr accessmsr { unsigned int cpu; unsigned int msr; unsigned long value; } do_sysctl () { ... case XEN_SYSCTL_accessmsr: /* store the msr value in accessmsr.value */ on_selected_cpus(cpumask_of(cpu), read_msr, &(op->u.accessmsr), 1); } Thanks, Dongxiao > > Jan > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel