From: Juergen Gross <jgross@suse.com> To: Jan Beulich <JBeulich@suse.com> Cc: Stefano Stabellini <sstabellini@kernel.org>, WeiLiu <wl@xen.org>, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>, George Dunlap <George.Dunlap@eu.citrix.com>, Andrew Cooper <andrew.cooper3@citrix.com>, Tim Deegan <tim@xen.org>, Dario Faggioli <dfaggioli@suse.com>, Julien Grall <julien.grall@arm.com>, xen-devel <xen-devel@lists.xenproject.org>, Ian Jackson <Ian.Jackson@eu.citrix.com>, Roger Pau Monne <roger.pau@citrix.com> Subject: Re: [PATCH 54/60] xen/sched: add minimalistic idle scheduler for free cpus Date: Tue, 28 May 2019 13:58:45 +0200 [thread overview] Message-ID: <5ea454f1-79f7-2974-0736-3262870446ab@suse.com> (raw) In-Reply-To: <5CED1FDC0200007800233141@prv1-mh.provo.novell.com> On 28/05/2019 13:47, Jan Beulich wrote: >>>> On 28.05.19 at 12:33, <jgross@suse.com> wrote: >> Instead of having a full blown scheduler running for the free cpus >> add a very minimalistic scheduler for that purpose only ever scheduling >> the related idle vcpu. This has the big advantage of not needing any >> per-cpu, per-domain or per-scheduling unit data for free cpus and in >> turn simplifying moving cpus to and from cpupools a lot. > > And the null scheduler is not minimalistic enough? The main disadvantage of the null scheduler are the need for private data (which has to be allocated/freed on scheduler switch), its not yet perfect stability, and its "complexity" (e.g. 900 lines vs. 50). > >> --- a/xen/arch/arm/smpboot.c >> +++ b/xen/arch/arm/smpboot.c >> @@ -350,8 +350,6 @@ void start_secondary(unsigned long boot_phys_offset, >> >> setup_cpu_sibling_map(cpuid); >> >> - scheduler_percpu_init(cpuid); >> - >> /* Run local notifiers */ >> notify_cpu_starting(cpuid); >> /* >> --- a/xen/arch/x86/smpboot.c >> +++ b/xen/arch/x86/smpboot.c >> @@ -382,8 +382,6 @@ void start_secondary(void *unused) >> >> set_cpu_sibling_map(cpu); >> >> - scheduler_percpu_init(cpu); >> - >> init_percpu_time(); >> >> setup_secondary_APIC_clock(); > > Seeing you revert here what an earlier patch in the series has added, > I don't suppose re-ordering could avoid this code churn? This would require another major shuffling of patches. I'd like to avoid that. Juergen _______________________________________________ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
WARNING: multiple messages have this Message-ID (diff)
From: Juergen Gross <jgross@suse.com> To: Jan Beulich <JBeulich@suse.com> Cc: Stefano Stabellini <sstabellini@kernel.org>, WeiLiu <wl@xen.org>, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>, George Dunlap <George.Dunlap@eu.citrix.com>, Andrew Cooper <andrew.cooper3@citrix.com>, Tim Deegan <tim@xen.org>, Dario Faggioli <dfaggioli@suse.com>, Julien Grall <julien.grall@arm.com>, xen-devel <xen-devel@lists.xenproject.org>, Ian Jackson <Ian.Jackson@eu.citrix.com>, Roger Pau Monne <roger.pau@citrix.com> Subject: Re: [Xen-devel] [PATCH 54/60] xen/sched: add minimalistic idle scheduler for free cpus Date: Tue, 28 May 2019 13:58:45 +0200 [thread overview] Message-ID: <5ea454f1-79f7-2974-0736-3262870446ab@suse.com> (raw) Message-ID: <20190528115845.8Z4oEclhKhzawLLmqGwy_j4HcoH1IDeOpdwzuAv4ZZI@z> (raw) In-Reply-To: <5CED1FDC0200007800233141@prv1-mh.provo.novell.com> On 28/05/2019 13:47, Jan Beulich wrote: >>>> On 28.05.19 at 12:33, <jgross@suse.com> wrote: >> Instead of having a full blown scheduler running for the free cpus >> add a very minimalistic scheduler for that purpose only ever scheduling >> the related idle vcpu. This has the big advantage of not needing any >> per-cpu, per-domain or per-scheduling unit data for free cpus and in >> turn simplifying moving cpus to and from cpupools a lot. > > And the null scheduler is not minimalistic enough? The main disadvantage of the null scheduler are the need for private data (which has to be allocated/freed on scheduler switch), its not yet perfect stability, and its "complexity" (e.g. 900 lines vs. 50). > >> --- a/xen/arch/arm/smpboot.c >> +++ b/xen/arch/arm/smpboot.c >> @@ -350,8 +350,6 @@ void start_secondary(unsigned long boot_phys_offset, >> >> setup_cpu_sibling_map(cpuid); >> >> - scheduler_percpu_init(cpuid); >> - >> /* Run local notifiers */ >> notify_cpu_starting(cpuid); >> /* >> --- a/xen/arch/x86/smpboot.c >> +++ b/xen/arch/x86/smpboot.c >> @@ -382,8 +382,6 @@ void start_secondary(void *unused) >> >> set_cpu_sibling_map(cpu); >> >> - scheduler_percpu_init(cpu); >> - >> init_percpu_time(); >> >> setup_secondary_APIC_clock(); > > Seeing you revert here what an earlier patch in the series has added, > I don't suppose re-ordering could avoid this code churn? This would require another major shuffling of patches. I'd like to avoid that. Juergen _______________________________________________ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
next prev parent reply other threads:[~2019-05-28 11:58 UTC|newest] Thread overview: 202+ messages / expand[flat|nested] mbox.gz Atom feed top 2019-05-28 10:32 [PATCH 00/60] xen: add core scheduling support Juergen Gross 2019-05-28 10:32 ` [Xen-devel] " Juergen Gross 2019-05-28 10:32 ` [PATCH 01/60] xen/sched: only allow schedulers with all mandatory functions available Juergen Gross 2019-05-28 10:32 ` [Xen-devel] " Juergen Gross 2019-06-11 16:03 ` Dario Faggioli 2019-05-28 10:32 ` [PATCH 02/60] xen/sched: add inline wrappers for calling per-scheduler functions Juergen Gross 2019-05-28 10:32 ` [Xen-devel] " Juergen Gross 2019-06-11 16:21 ` Dario Faggioli 2019-05-28 10:32 ` [PATCH 03/60] xen/sched: let sched_switch_sched() return new lock address Juergen Gross 2019-05-28 10:32 ` [Xen-devel] " Juergen Gross 2019-06-11 16:55 ` Dario Faggioli 2019-06-12 7:40 ` Jan Beulich 2019-06-12 8:06 ` Juergen Gross 2019-06-12 9:44 ` Dario Faggioli 2019-06-12 8:05 ` Andrew Cooper 2019-06-12 8:19 ` Juergen Gross 2019-06-12 9:32 ` Jan Beulich [not found] ` <5D00C6960200007800237622@suse.com> 2019-06-12 9:56 ` Dario Faggioli 2019-06-12 10:14 ` Jan Beulich 2019-06-12 11:27 ` Andrew Cooper 2019-06-12 13:32 ` Dario Faggioli 2019-05-28 10:32 ` [PATCH 04/60] xen/sched: use new sched_unit instead of vcpu in scheduler interfaces Juergen Gross 2019-05-28 10:32 ` [Xen-devel] " Juergen Gross 2019-07-18 17:44 ` Dario Faggioli 2019-07-19 4:49 ` Juergen Gross 2019-07-19 17:01 ` Dario Faggioli 2019-05-28 10:32 ` [PATCH 05/60] xen/sched: alloc struct sched_unit for each vcpu Juergen Gross 2019-05-28 10:32 ` [Xen-devel] " Juergen Gross 2019-07-18 17:57 ` Dario Faggioli 2019-07-19 4:56 ` Juergen Gross 2019-07-19 17:04 ` Dario Faggioli 2019-05-28 10:32 ` [PATCH 06/60] xen/sched: move per-vcpu scheduler private data pointer to sched_unit Juergen Gross 2019-05-28 10:32 ` [Xen-devel] " Juergen Gross 2019-07-18 22:52 ` Dario Faggioli 2019-07-19 5:03 ` Juergen Gross 2019-07-19 17:10 ` Dario Faggioli 2019-05-28 10:32 ` [PATCH 07/60] xen/sched: build a linked list of struct sched_unit Juergen Gross 2019-05-28 10:32 ` [Xen-devel] " Juergen Gross 2019-07-19 0:01 ` Dario Faggioli 2019-07-19 5:07 ` Juergen Gross 2019-07-19 17:16 ` Dario Faggioli 2019-05-28 10:32 ` [PATCH 08/60] xen/sched: introduce struct sched_resource Juergen Gross 2019-05-28 10:32 ` [Xen-devel] " Juergen Gross 2019-07-19 17:43 ` Dario Faggioli 2019-07-19 17:49 ` Dario Faggioli 2019-05-28 10:32 ` [PATCH 09/60] xen/sched: let pick_cpu return a scheduler resource Juergen Gross 2019-05-28 10:32 ` [Xen-devel] " Juergen Gross 2019-07-19 18:06 ` Dario Faggioli 2019-05-28 10:32 ` [PATCH 10/60] xen/sched: switch schedule_data.curr to point at sched_unit Juergen Gross 2019-05-28 10:32 ` [Xen-devel] " Juergen Gross 2019-07-29 22:08 ` Dario Faggioli 2019-05-28 10:32 ` [PATCH 11/60] xen/sched: move per cpu scheduler private data into struct sched_resource Juergen Gross 2019-05-28 10:32 ` [Xen-devel] " Juergen Gross 2019-05-28 11:32 ` Jan Beulich 2019-05-28 11:32 ` [Xen-devel] " Jan Beulich 2019-07-29 22:22 ` Dario Faggioli 2019-05-28 10:32 ` [PATCH 12/60] xen/sched: switch vcpu_schedule_lock to unit_schedule_lock Juergen Gross 2019-05-28 10:32 ` [Xen-devel] " Juergen Gross 2019-05-28 10:32 ` [PATCH 13/60] xen/sched: move some per-vcpu items to struct sched_unit Juergen Gross 2019-05-28 10:32 ` [Xen-devel] " Juergen Gross 2019-06-13 7:18 ` Andrii Anisov 2019-06-13 7:29 ` Juergen Gross 2019-06-13 7:34 ` Andrii Anisov 2019-06-13 8:39 ` Juergen Gross 2019-06-13 8:49 ` Andrii Anisov 2019-05-28 10:32 ` [PATCH 14/60] xen/sched: add scheduler helpers hiding vcpu Juergen Gross 2019-05-28 10:32 ` [Xen-devel] " Juergen Gross 2019-05-28 10:32 ` [PATCH 15/60] xen/sched: add domain pointer to struct sched_unit Juergen Gross 2019-05-28 10:32 ` [Xen-devel] " Juergen Gross 2019-05-28 10:32 ` [PATCH 16/60] xen/sched: add id " Juergen Gross 2019-05-28 10:32 ` [Xen-devel] " Juergen Gross 2019-05-28 10:32 ` [PATCH 17/60] xen/sched: rename scheduler related perf counters Juergen Gross 2019-05-28 10:32 ` [Xen-devel] " Juergen Gross 2019-05-28 10:32 ` [PATCH 18/60] xen/sched: switch struct task_slice from vcpu to sched_unit Juergen Gross 2019-05-28 10:32 ` [Xen-devel] " Juergen Gross 2019-05-28 10:32 ` [PATCH 19/60] xen/sched: add is_running indicator to struct sched_unit Juergen Gross 2019-05-28 10:32 ` [Xen-devel] " Juergen Gross 2019-05-28 10:32 ` [PATCH 20/60] xen/sched: make null scheduler vcpu agnostic Juergen Gross 2019-05-28 10:32 ` [Xen-devel] " Juergen Gross 2019-05-28 10:32 ` [PATCH 21/60] xen/sched: make rt " Juergen Gross 2019-05-28 10:32 ` [Xen-devel] " Juergen Gross 2019-05-28 10:32 ` [PATCH 22/60] xen/sched: make credit " Juergen Gross 2019-05-28 10:32 ` [Xen-devel] " Juergen Gross 2019-05-28 10:32 ` [PATCH 23/60] xen/sched: make credit2 " Juergen Gross 2019-05-28 10:32 ` [Xen-devel] " Juergen Gross 2019-05-28 10:32 ` [PATCH 24/60] xen/sched: make arinc653 " Juergen Gross 2019-05-28 10:32 ` [Xen-devel] " Juergen Gross 2019-05-28 10:32 ` [PATCH 25/60] xen: add sched_unit_pause_nosync() and sched_unit_unpause() Juergen Gross 2019-05-28 10:32 ` [Xen-devel] " Juergen Gross 2019-05-28 10:32 ` [PATCH 26/60] xen: let vcpu_create() select processor Juergen Gross 2019-05-28 10:32 ` [Xen-devel] " Juergen Gross 2019-05-28 10:32 ` [PATCH 27/60] xen/sched: use sched_resource cpu instead smp_processor_id in schedulers Juergen Gross 2019-05-28 10:32 ` [Xen-devel] " Juergen Gross 2019-05-28 10:32 ` [PATCH 28/60] xen/sched: switch schedule() from vcpus to sched_units Juergen Gross 2019-05-28 10:32 ` [Xen-devel] " Juergen Gross 2019-05-28 10:32 ` [PATCH 29/60] xen/sched: switch sched_move_irqs() to take sched_unit as parameter Juergen Gross 2019-05-28 10:32 ` [Xen-devel] " Juergen Gross 2019-05-28 10:32 ` [PATCH 30/60] xen: switch from for_each_vcpu() to for_each_sched_unit() Juergen Gross 2019-05-28 10:32 ` [Xen-devel] " Juergen Gross 2019-05-28 10:32 ` [PATCH 31/60] xen/sched: add runstate counters to struct sched_unit Juergen Gross 2019-05-28 10:32 ` [Xen-devel] " Juergen Gross 2019-05-28 10:32 ` [PATCH 32/60] xen/sched: rework and rename vcpu_force_reschedule() Juergen Gross 2019-05-28 10:32 ` [Xen-devel] " Juergen Gross 2019-05-28 10:32 ` [PATCH 33/60] xen/sched: Change vcpu_migrate_*() to operate on schedule unit Juergen Gross 2019-05-28 10:32 ` [Xen-devel] " Juergen Gross 2019-05-28 10:32 ` [PATCH 34/60] xen/sched: move struct task_slice into struct sched_unit Juergen Gross 2019-05-28 10:32 ` [Xen-devel] " Juergen Gross 2019-05-28 10:32 ` [PATCH 35/60] xen/sched: add code to sync scheduling of all vcpus of a sched unit Juergen Gross 2019-05-28 10:32 ` [Xen-devel] " Juergen Gross 2019-05-28 10:32 ` [PATCH 36/60] xen/sched: introduce unit_runnable_state() Juergen Gross 2019-05-28 10:32 ` [Xen-devel] " Juergen Gross 2019-05-28 10:32 ` [PATCH 37/60] xen/sched: add support for multiple vcpus per sched unit where missing Juergen Gross 2019-05-28 10:32 ` [Xen-devel] " Juergen Gross 2019-05-28 10:32 ` [PATCH 38/60] x86: make loading of GDT at context switch more modular Juergen Gross 2019-05-28 10:32 ` [Xen-devel] " Juergen Gross 2019-07-02 15:38 ` Andrew Cooper 2019-05-28 10:32 ` [PATCH 39/60] x86: optimize loading of GDT at context switch Juergen Gross 2019-05-28 10:32 ` [Xen-devel] " Juergen Gross 2019-07-02 16:09 ` Andrew Cooper 2019-07-03 6:30 ` Juergen Gross 2019-07-03 12:21 ` Andrew Cooper 2019-07-05 7:30 ` Juergen Gross 2019-05-28 10:32 ` [PATCH 40/60] xen/sched: modify cpupool_domain_cpumask() to be an unit mask Juergen Gross 2019-05-28 10:32 ` [Xen-devel] " Juergen Gross 2019-05-28 10:32 ` [PATCH 41/60] xen/sched: support allocating multiple vcpus into one sched unit Juergen Gross 2019-05-28 10:32 ` [Xen-devel] " Juergen Gross 2019-05-28 10:32 ` [PATCH 42/60] xen/sched: add a scheduler_percpu_init() function Juergen Gross 2019-05-28 10:32 ` [Xen-devel] " Juergen Gross 2019-05-28 10:32 ` [PATCH 43/60] xen/sched: add a percpu resource index Juergen Gross 2019-05-28 10:32 ` [Xen-devel] " Juergen Gross 2019-05-28 10:32 ` [PATCH 44/60] xen/sched: add fall back to idle vcpu when scheduling unit Juergen Gross 2019-05-28 10:32 ` [Xen-devel] " Juergen Gross 2019-05-28 10:32 ` [PATCH 45/60] xen/sched: make vcpu_wake() and vcpu_sleep() core scheduling aware Juergen Gross 2019-05-28 10:32 ` [Xen-devel] " Juergen Gross 2019-05-28 10:32 ` [PATCH 46/60] xen/sched: carve out freeing sched_unit memory into dedicated function Juergen Gross 2019-05-28 10:32 ` [Xen-devel] " Juergen Gross 2019-05-28 10:33 ` [PATCH 47/60] xen/sched: move per-cpu variable scheduler to struct sched_resource Juergen Gross 2019-05-28 10:33 ` [Xen-devel] " Juergen Gross 2019-05-28 10:33 ` [PATCH 48/60] xen/sched: move per-cpu variable cpupool " Juergen Gross 2019-05-28 10:33 ` [Xen-devel] " Juergen Gross 2019-05-28 10:33 ` [PATCH 49/60] xen/sched: reject switching smt on/off with core scheduling active Juergen Gross 2019-05-28 10:33 ` [Xen-devel] " Juergen Gross 2019-05-28 11:44 ` Jan Beulich 2019-05-28 11:44 ` [Xen-devel] " Jan Beulich 2019-05-28 11:52 ` Juergen Gross 2019-05-28 11:52 ` [Xen-devel] " Juergen Gross 2019-06-12 9:36 ` Dario Faggioli 2019-05-28 10:33 ` [PATCH 50/60] xen/sched: prepare per-cpupool scheduling granularity Juergen Gross 2019-05-28 10:33 ` [Xen-devel] " Juergen Gross 2019-05-28 10:33 ` [PATCH 51/60] xen/sched: use one schedule lock for all free cpus Juergen Gross 2019-05-28 10:33 ` [Xen-devel] " Juergen Gross 2019-05-28 10:33 ` [PATCH 52/60] xen/sched: populate cpupool0 only after all cpus are up Juergen Gross 2019-05-28 10:33 ` [Xen-devel] " Juergen Gross 2019-05-28 10:33 ` [PATCH 53/60] xen/sched: remove cpu from pool0 before removing it Juergen Gross 2019-05-28 10:33 ` [Xen-devel] " Juergen Gross 2019-05-28 10:33 ` [PATCH 54/60] xen/sched: add minimalistic idle scheduler for free cpus Juergen Gross 2019-05-28 10:33 ` [Xen-devel] " Juergen Gross 2019-05-28 11:47 ` Jan Beulich 2019-05-28 11:47 ` [Xen-devel] " Jan Beulich 2019-05-28 11:58 ` Juergen Gross [this message] 2019-05-28 11:58 ` Juergen Gross 2019-05-31 14:15 ` Dario Faggioli 2019-05-31 14:15 ` [Xen-devel] " Dario Faggioli 2019-05-31 15:52 ` Dario Faggioli 2019-05-31 15:52 ` [Xen-devel] " Dario Faggioli 2019-05-31 16:44 ` Juergen Gross 2019-05-31 16:44 ` [Xen-devel] " Juergen Gross 2019-05-28 10:33 ` [PATCH 55/60] xen/sched: split schedule_cpu_switch() Juergen Gross 2019-05-28 10:33 ` [Xen-devel] " Juergen Gross 2019-05-28 10:33 ` [PATCH 56/60] xen/sched: protect scheduling resource via rcu Juergen Gross 2019-05-28 10:33 ` [Xen-devel] " Juergen Gross 2019-05-28 10:33 ` [PATCH 57/60] xen/sched: support multiple cpus per scheduling resource Juergen Gross 2019-05-28 10:33 ` [Xen-devel] " Juergen Gross 2019-05-28 10:33 ` [PATCH 58/60] xen/sched: support differing granularity in schedule_cpu_[add/rm]() Juergen Gross 2019-05-28 10:33 ` [Xen-devel] " Juergen Gross 2019-05-28 10:33 ` [PATCH 59/60] xen/sched: support core scheduling for moving cpus to/from cpupools Juergen Gross 2019-05-28 10:33 ` [Xen-devel] " Juergen Gross 2019-05-28 10:33 ` [PATCH 60/60] xen/sched: add scheduling granularity enum Juergen Gross 2019-05-28 10:33 ` [Xen-devel] " Juergen Gross 2019-05-28 11:51 ` Jan Beulich 2019-05-28 11:51 ` [Xen-devel] " Jan Beulich 2019-05-28 12:02 ` Juergen Gross 2019-05-28 12:02 ` [Xen-devel] " Juergen Gross 2019-07-19 18:31 ` Dario Faggioli 2019-07-05 13:17 ` [Xen-devel] [PATCH 00/60] xen: add core scheduling support Sergey Dyasli 2019-07-05 13:22 ` Juergen Gross 2019-07-05 13:56 ` Dario Faggioli 2019-07-15 14:08 ` Sergey Dyasli 2019-07-18 14:48 ` Juergen Gross 2019-07-18 15:14 ` Sergey Dyasli 2019-07-18 16:04 ` Dario Faggioli 2019-07-19 5:41 ` Juergen Gross 2019-07-19 11:24 ` Juergen Gross 2019-07-19 13:57 ` Juergen Gross 2019-07-22 14:22 ` Sergey Dyasli 2019-07-24 9:13 ` Juergen Gross 2019-07-24 14:54 ` Sergey Dyasli 2019-07-24 15:11 ` Juergen Gross 2019-07-16 15:45 ` Sergey Dyasli 2019-07-19 13:35 ` Juergen Gross 2019-07-25 16:01 ` Sergey Dyasli 2019-07-11 13:40 ` Dario Faggioli [not found] <20190528103313.13431jgross@suse.com> [not found] ` <20190528103313.13434jgross@suse.com> [not found] <20190528103313.13431jgross@suse.com> [not found] ` <20190528103313.13434jgross@suse.com>
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=5ea454f1-79f7-2974-0736-3262870446ab@suse.com \ --to=jgross@suse.com \ --cc=George.Dunlap@eu.citrix.com \ --cc=Ian.Jackson@eu.citrix.com \ --cc=JBeulich@suse.com \ --cc=andrew.cooper3@citrix.com \ --cc=dfaggioli@suse.com \ --cc=julien.grall@arm.com \ --cc=konrad.wilk@oracle.com \ --cc=roger.pau@citrix.com \ --cc=sstabellini@kernel.org \ --cc=tim@xen.org \ --cc=wl@xen.org \ --cc=xen-devel@lists.xenproject.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).