Re: Xen Platform QoS design discussion

["Xu, Dongxiao" <dongxiao.xu@intel.com>]

> -----Original Message-----
> From: Andrew Cooper [mailto:andrew.cooper3@citrix.com]
> Sent: Friday, May 02, 2014 8:51 PM
> To: Xu, Dongxiao
> Cc: Jan Beulich; Ian Campbell; xen-devel@lists.xen.org
> Subject: Re: Xen Platform QoS design discussion
> 
> On 02/05/14 13:30, Xu, Dongxiao wrote:
> >> -----Original Message-----
> >> From: Jan Beulich [mailto:JBeulich@suse.com]
> >> Sent: Friday, May 02, 2014 5:24 PM
> >> To: Xu, Dongxiao
> >> Cc: Andrew Cooper(andrew.cooper3@citrix.com); Ian Campbell;
> >> xen-devel@lists.xen.org
> >> Subject: RE: Xen Platform QoS design discussion
> >>
> >>>>> On 01.05.14 at 02:56, <dongxiao.xu@intel.com> wrote:
> >>>> From: Ian Campbell [mailto:Ian.Campbell@citrix.com]
> >>>> Have you asked yourself whether this information even needs to be
> >>>> exposed all the way up to libxl? Who are the expected consumers of this
> >>>> interface? Are they low-level CLI tools (i.e. like xenpm is) or are you
> >>>> expecting toolstacks to plumb this information all the way up to their
> >>>> GUI or CLI (e.g. xl or virsh)?
> >>> The information returned to libxl users is the cache utilization for a
> >>> certain domain in certain socket, and the main consumers are cloud users
> like
> >>> openstack, etc. Of course, we will also provide an xl command to present
> such
> >>> information.
> >> To me this doesn't really address the question Ian asked, yet knowing
> >> who's going to be the consumer of the data is also quite relevant for
> >> answering your original question on the method to obtain that data.
> >> Obviously, if the main use of it is per-domain, a domctl would seem like
> >> a suitable approach despite the data being more of sysctl kind. But if
> >> a global view would be more important, that model would seem to make
> >> life needlessly hard for the consumers. In turn, if using a domctl, I tend
> >> to agree that not using shared pages would be preferable; iirc their use
> >> was mainly suggested because of the size of the data.
> > From the discussion with openstack developers, on certain cloud host, all
> running VM's information (e.g., domain ID) will be stored in a database, and
> openstack software will use libvirt/XenAPI to query specific domain information.
> That libvirt/XenAPI API interface basically accepts the domain ID as input
> parameter and get the domain information, including the platform QoS one.
> >
> > Based on above information, I think we'd better design the QoS hypercall
> per-domain.
> 
> The design of the hypercall has nothing to do with the design of the
> libxl/XenAPI interface.

If use the share mechanism between Xen and Dom0 user space, plus explicitly listing all the available CQM features as you proposed (see below structure cited from previous mail), then the ABI between Xen and Dom0 user space may need to be changing every time when a new QoS feature is introduced, which breaks the compatibility to some extent. :(

struct
{
    uint64_t[nr l3 events xen knows how to collect] l3_data;
    uint... new categories.
} [system max_rmid];


uint64_t[max_cqm_rmid] l3_occupency;
uint64_t[max_mbm_rmid] l3_total_bandwidth;
uint64_t[max_mbm_rmid] l3_local_bandwidth;
uint... new categories.

Thanks,
Dongxiao

> 
> It is clear from this statement that cloudstack want all information for
> all domains.  Therefore, at one level at least there will be a big set
> of nested loops like:
> 
> every $TIMEPERIOD
>   for each domain
>     for each type of information
>       get-$TYPE-information-for-$DOMAIN
> 
> As far as a XenAPI inteface would go, this would be information coming
> from the rrdd-daemon.  This daemon most certainly wont want to be using
> a hypercall designed like this.
> 
> Does anyone know how libvirt/libxl would go about
> collecting/storing/passing this information?
> 
> ~Andrew