From mboxrd@z Thu Jan 1 00:00:00 1970 From: Chong Li Subject: Re: [PATCH v2 for Xen 4.6 3/4] libxl: enabling XL to set per-VCPU parameters of a domain for RTDS scheduler Date: Fri, 12 Jun 2015 15:48:52 -0500 Message-ID: References: <1432598984-20914-1-git-send-email-chong.li@wustl.edu> <1433504253.7108.231.camel@citrix.com> <1433778984.2403.27.camel@citrix.com> <1433866738.2403.181.camel@citrix.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <1433866738.2403.181.camel@citrix.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Dario Faggioli Cc: Chong Li , Wei Liu , Ian Campbell , Sisu Xi , George Dunlap , Ian Jackson , xen-devel , Meng Xu , Dagaen Golomb List-Id: xen-devel@lists.xenproject.org If no more feedbacks, let me summarize the design for the next version. For "get" operations, we will implement the following features: 1) Use " xl sched-rtds -v all " to output the per-dom parameters of all domains. And use, e.g., " xl sched-rtds -d vm1 -v all ", to output the per-dom parameters of one specific domain. When a domain (say vm1) has vcpus with different scheduling parameters but meanwhile the user uses "xl sched-rtds -d vm1 -v all " to show the per-dom parameters, the actual output result is just the parameters of vcpu with ID=0 (which is pointless, and should be made clear to the users). These two kinds of "get" operations would be implemented through libxl_domain_sched_params_get() and other domain-related functions (no changes to all these functions). 2) For example, use " xl sched-rtds -d vm1 -v 0 -v 2 -v 4 " to show the per-vcpu parameters of vcpu"0", vcpu"2" and vcpu"4" of vm1. This kind of "get" operation would be implemented through libxl_vcpu_sched_params_get() and other newly-added vcpu-related functions. For "set" operations, no new feature is added, against patch v2. We need some new data structures to support per-vcpu operations (for all schedulers, not just RTDS). 1) In libxl, we will introduce: libxl_vcpu_sched_params = Struct("vcpu_sched_params",[ ("vcpuid", integer, { xxx some init val xxx}), ("weight", integer, {'init_val': 'LIBXL_PARAM_WEIGHT_DEFAULT'}), ("cap", integer, {'init_val': 'LIBXL_PARAM_CAP_DEFAULT'}), ("period", integer, {'init_val': 'LIBXL_PARAM_PERIOD_DEFAULT'}), ("slice", integer, {'init_val': 'LIBXL_PARAM_SLICE_DEFAULT'}), ("latency", integer, {'init_val': 'LIBXL_PARAM_LATENCY_DEFAULT'}), ("extratime", integer, {'init_val': 'LIBXL_PARAM_EXTRATIME_DEFAULT'}), ("budget", integer, {'init_val': 'LIBXL_PARAM_BUDGET_DEFAULT'}), ]) libxl_sched_params = Struct("sched_params",[ ("sched", libxl_scheduler), ("vcpus", Array(libxl_sched_params, "num_vcpus")), ]) and use libxl_sched_params to store and transfer vcpu array with parameters to change/output. 2) In xen, we will introduce: struct xen_domctl_scheduler_op { uint32_t sched_id; /* XEN_SCHEDULER_* */ uint32_t cmd; /* XEN_DOMCTL_SCHEDOP_* */ union { xen_domctl_schedparam_t d; struct { XEN_GUEST_HANDLE_64(xen_domctl_schedparam_vcpu_t) vcpus; uint16_t nr_vcpus; } v; } u; }; typedef struct xen_domctl_scheduler_op xen_domctl_scheduler_op_t; DEFINE_XEN_GUEST_HANDLE(xen_domctl_scheduler_op_t); and some others (details can be found in http://www.gossamer-threads.com/lists/xen/devel/380726?do=post_view_threaded ). Because of this new xen_domctl_scheduler_op_t, some changes have to be done for credit and credit2 schedulers (for the XEN_DOMCTL_scheduler_op processing there). Please correct me if something is wrong. Thanks, Chong On Tue, Jun 9, 2015 at 11:18 AM, Dario Faggioli wrote: > On Mon, 2015-06-08 at 15:55 -0500, Chong Li wrote: >> On Mon, Jun 8, 2015 at 10:56 AM, Dario Faggioli > >> > So, Thoughts? What do you think the best way forward could be? >> >> I like option 2 more. But I think we may also need a 'vcpuid' field in >> libxl_sched_params. >> > For sparse array support, yes. At which point, I would flip the names as > well, i.e., something like this: > > libxl_vcpu_sched_params = Struct("vcpu_sched_params",[ > ("vcpuid", integer, { xxx some init val xxx}), > ("weight", integer, {'init_val': 'LIBXL_PARAM_WEIGHT_DEFAULT'}), > ("cap", integer, {'init_val': 'LIBXL_PARAM_CAP_DEFAULT'}), > ("period", integer, {'init_val': 'LIBXL_PARAM_PERIOD_DEFAULT'}), > ("slice", integer, {'init_val': 'LIBXL_PARAM_SLICE_DEFAULT'}), > ("latency", integer, {'init_val': 'LIBXL_PARAM_LATENCY_DEFAULT'}), > ("extratime", integer, {'init_val': 'LIBXL_PARAM_EXTRATIME_DEFAULT'}), > ("budget", integer, {'init_val': 'LIBXL_PARAM_BUDGET_DEFAULT'}), > ]) > > libxl_sched_params = Struct("sched_params",[ > ("sched", libxl_scheduler), > ("vcpus", Array(libxl_sched_params, "num_vcpus")), > ]) > > With the possibility of naming the latter 'libxl_vcpus_sched_params', > which is more descriptive, but perhaps is too similar to > libxl_vcpu_sched_params. > > Ian, George, what do you think? > > While we're here, another thing we would appreciate some feedback on is > what should happen to libxl_domain_sched_params_get(). This occurred to > my mind while reviewing patch 4 of this series. Actually, I think we've > discussed this before, but can't find the reference now. > > Anyway, my view is that, for a scheduler that uses per-vcpu parameters, > libxl_domain_sched_params_set() should set the same parameters for all > the vcpus. > When it comes to _get(), however, I'm not sure. To match the _set() > case, we'd need to return the parameters of all the vcpus, but we can't, > because the function takes a libxl_domain_sched_params argument, which > just holds 1 tuple. > > Should we just WARN and ask, when on that specific scheduler, to use the > per-vcpu variant being introduced in this patch > (libxl_vcpu_sched_params_get())? > > This does not look ideal, but without changing the prototype of > libxl_domain_sched_params_get(), I don't see what else sensible we could > do... :-/ > > Should we change it, and do the LIBXL_API_VERSION "trick"? > > So, again, thoughts? > > Regards, > Dario > > -- > <> (Raistlin Majere) > ----------------------------------------------------------------- > Dario Faggioli, Ph.D, http://about.me/dario.faggioli > Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK) -- Chong Li Department of Computer Science and Engineering Washington University in St.louis