-----Original Message----- From: linux-kernel-owner@vger.kernel.org On Behalf Of Vineeth Remanan Pillai Sent: Friday, January 17, 2020 8:01 AM To: Aubrey Li Cc: Tim Chen ; Aubrey Li ; Nishanth Aravamudan ; Julien Desfossez ; Peter Zijlstra ; Ingo Molnar ; Thomas Gleixner ; Paul Turner ; Linus Torvalds ; Linux List Kernel Mailing ; Dario Faggioli ; Frédéric Weisbecker ; Kees Cook ; Greg Kerr ; Phil Auld ; Aaron Lu ; Valentin Schneider ; Mel Gorman ; Pawan Gupta ; Paolo Bonzini Subject: Re: [RFC PATCH v4 00/19] Core scheduling v4 > > > > Aubrey's attached patch should replace his previous patch > > sched/fair: don't migrate task if cookie not match > > > > I've also added a fix below for Aubrey's patch > > sched/fair: find cookie matched idlest CPU. > > > > Aubrey, can you merge this fix into that patch when you update your > > patches? > > Thanks Tim, I'll include your fix when I update my patches. > We have included both these changes in our branch rebased to 5.4.y: https://github.com/digitalocean/linux-coresched/tree/coresched/v4-v5.4.y This is our testing branch. ---------------------------------------------------------------------- ABOUT: ---------------------------------------------------------------------- Hello, Core scheduling is required to protect against leakage of sensitive data allocated on a sibling thread. Our goal is to measure performance impact of core scheduling across different workloads and show how it evolved over time. Below you will find data for core scheduling. Data presented here are based on previous version of core-sched (v3) plus additional kernel patches added by tim.c.chen@intel.com and load balancer made by aubrey.li@intel.com that are now in (v4) core scheduler kernel https://github.com/digitalocean/linux-coresched/commits/coresched/v4-v5.4.y Detailed specifics about each system set up in attached PDF. ---------------------------------------------------------------------- BENCHMARKS: ---------------------------------------------------------------------- - SPECreate2017int : industry standard for measure compute intensive CPU performance. - CASSANDRA : NoSQL distributed DB and benchmark. Provides high availability with no single point of failure. - HammerDB : database benchmarking application - SPECvirt : industry standard for measure server performance in virtual environment. ---------------------------------------------------------------------- PERFORMANCE IMPACT: ---------------------------------------------------------------------- --------------------------------------------------------------------------------------------------------------------------------------------------- Kernel & Benchmark |# of cgroups|Overcommit | Performance Change %| Performance Change % | | | (HT ON, core-sched) | (HT OFF, No core-sched) | --------------------------------------------------------------------------------------------------------------------------------------------------- | 2cgroups | 0.5x | -0.48 | 0.26 | SPECint_rate_base 2017 | 1 per VM | 1x | -0.54 | 0.66 | | | 2x | -0.18 | -0.07 | --------------------------------------------------------------------------------------------------------------------------------------------------- | 2cgroups | 0.5x | -2.56 | -44.61 | CASSANDRA | 1 per VM | 1x | -4.71 | -47.47 | | | 2x | -7.63 | -28.47 | --------------------------------------------------------------------------------------------------------------------------------------------------- | 2cgroups | 0.5x | -19.99 | -23.52 | HammerDB | 1 per VM | 1x | -14.90 | -24.63 | | | 2x | -13.31 | -25.71 | --------------------------------------------------------------------------------------------------------------------------------------------------- | attachment |attachment | -18 | -31 | SPECvirt | | | | | --------------------------------------------------------------------------------------------------------------------------------------------------- | 2cgroups | 0.5x | -5.18 | -44.61 | CASSANDRA | 1 per VM | 1x | -7.90 | -47.47 | | | 2x | -6.52 | -28.47 | --------------------------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------- TAKE AWAYS: ---------------------------------------------------------------------- 1. Core scheduling performs better than turning off HT in all overcommitting scenarios. However we observed in certain cases up to 20% performance drop. 2. Impact of core scheduling depends on the workload and thread scheduling intensity. 3. Core scheduling requires cgroups. Single cgroup per VM. Each VM is running on it’s own independent cgroup. Many thanks, --Agata