From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1431817AbdDYVQs (ORCPT ); Tue, 25 Apr 2017 17:16:48 -0400 Received: from mx0b-00082601.pphosted.com ([67.231.153.30]:46111 "EHLO mx0b-00082601.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1431738AbdDYVQj (ORCPT ); Tue, 25 Apr 2017 17:16:39 -0400 Subject: Re: [PATCH 2/2] sched/fair: Always propagate runnable_load_avg To: Tejun Heo , Vincent Guittot References: <20170424201344.GA14169@wtj.duckdns.org> <20170424201444.GC14169@wtj.duckdns.org> <20170425184941.GB15593@wtj.duckdns.org> <20170425204907.GA20255@wtj.duckdns.org> CC: Ingo Molnar , Peter Zijlstra , linux-kernel , Linus Torvalds , Mike Galbraith , Paul Turner , From: Chris Mason Message-ID: <6248cdfe-c5d1-35f2-003b-3a260b5a94ea@fb.com> Date: Tue, 25 Apr 2017 17:15:32 -0400 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 In-Reply-To: <20170425204907.GA20255@wtj.duckdns.org> Content-Type: text/plain; charset="windows-1252"; format=flowed Content-Transfer-Encoding: 7bit X-Originating-IP: [2620:10d:c091:180::1:f3f0] X-ClientProxiedBy: DM3PR12CA0074.namprd12.prod.outlook.com (10.161.151.146) To DM5PR15MB1660.namprd15.prod.outlook.com (10.175.108.151) X-MS-Office365-Filtering-Correlation-Id: d00d7d66-5718-494b-a677-08d48c2037f7 X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:(22001)(201703131423075)(201703031133081);SRVR:DM5PR15MB1660; X-Microsoft-Exchange-Diagnostics: 1;DM5PR15MB1660;3:l73Tc2anfohvFw1gXEXyh+eV2BAnibWBTIL4dQbxugdMA0hD/0rFyiM70rnYFsygpnECnHRRf2Uv51OWCDabMMXbX9gMbYO9qAPdRGIoi5bjrulmKbLduNALkIYMIO74+gHp6DAaEWCuq6IXSzcEOeG0+yhCpgtFay/hZUfPQLrqqAUbieoC0hNv8Y14OPpG6TGG+xpwmDHUfebFn2NI1GwRZoGdXIqcoQlaNAvVefo7T7u5fRYTrOs59jHeXQJrKuXEJ/lQpRe9eZdc3q6FYqP3qnux3xdlFQdNRMkjugwfoMBkdIuSNjK8guw1Z/TEpHtdAFDcd8M/jIpDkIX7/A==;25:BN38qscnP2uYug8cWEswUoB65VBY5eRgwp1o6W9+xm/yB4k8JH9K7sE3161lWD9Nl9aHLIkxlmrJPR+/rJUOPTop2diatqvOY1E7zF5d9M9xApf9pXo2q7HOd7X/jC1fvTXGB7sKMSIZHndw1bcXkv+5WbJFcd7ewnq34ZVOgn68NDtKjRrRvF5gWSL1HyTnG8dhlpGqukFAdBn+YdiXn6LzjmwpZA25uiYw9jKl31ZNTxXof5veRrsBMpKk6/tvESssXmETG0osWjbwjb7173pWDzKwlYYvLA1jMWbirxiSwUaMOG6fVcyAOCS8IjGWhyvQpRGd6+wTvMY0Kgo2oV2aHyoeW0MPb1tYBqPV8Sox8ySW8vajPp2Ckn5Fsl7lr8rSca+PgxmvY+4roqJ5Jb5VYAS6y3r9huWcyDl356xEJ6X2LisJY6Sdv0ZYTlC9trhkoe3+3YkjEDcd8aq3bQ== X-Microsoft-Exchange-Diagnostics: 1;DM5PR15MB1660;31:jH4Z3QAeUCf2zkasPusXmF4H46AppMv7UQYcZ/ZmL/7cI90z4MQU597zVl5tF4ZhP2HooZktvOquMwOhHg/hfhoNVnoQUwgnEYdwo5cvxbqpPPVAnBHNGAsYPjtsk5SSOurLwHZZl+qbveMqpTotkjnwF4JsokfgT6cdpBoxgHH+82A726gYYfUUUiz7pa6mqP0YFPP2DKUmFqVN8VNuAQHw8qLTXMboJx6JaTS7DhIzCtO78N1T5dPFAeaMC3fl7qmttl7gN1QAkYcuIvlYQQ==;20:vw+v+JVguxfbmaimrH2cKVwfFVAtYVfcEGsX4kX+MyeJgjThNlwdLg4ivGakJcmFof4eW7G98MlkoMizHetAOhhuiX8oXWFeZCA3MCnOTyrPFdVRKz6Box6aADVzDsiMQ7EKTiSsqI3HVRPnaL8jQdFfXKnC9Gj7GSgBIR4B/PoQ7GroCbYGUsdUJ9+0QguZvfeUPw41NlBKg4P7pBe8hzdQ6JPdz6LDOzenvtyyv52/szeHkQFSrLbJSXASN7AbKQw7FuwZSkNu87YdDsAPJKz8rg+zxk07rZFuqANfrMTPGzMyUc/2AaHLu5xf6pBYawAZo9ogomwlcBopRwjm0TA073QsRYOKt7/HhV8YxyZUpwSXQKwlgQUmWjLl2is0M+nKGU7l9IEhs0ZJGkZAJsYprb4WF1Oivb9qh5wJiCkc8JmNWaiZ9jRKapfRHUYCM0GgCK3trrdWvkDQTHLFSlB9sUm7u6WQAe1tKnfx3Il3ChF9urUjtW/qRKr+h+vB X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(6040450)(601004)(2401047)(8121501046)(5005006)(93006095)(93001095)(10201501046)(3002001)(6041248)(20161123560025)(20161123555025)(20161123564025)(201703131423075)(201702281528075)(201703061421075)(20161123562025)(6072148);SRVR:DM5PR15MB1660;BCL:0;PCL:0;RULEID:;SRVR:DM5PR15MB1660; X-Microsoft-Exchange-Diagnostics: 1;DM5PR15MB1660;4:v4Ckkb2k1nK2AM098gkMtSvWOjHjk6AhIMcaDQYL0nxw2aSdYfr5gpjC37YLCga8J6hDBVWOPPC2caPKAp9WoFHNmTMYS6BQhzfhEFHKqx8rShn9BAUegPHxO71AhTVNIA55g4TZZEDI/7XoAFArnr+XtujNO6Cjy1mqEssSlHX2OrtUIVxghly5Zi+diQUbTZiYW7hL1HurQEyEalLNehjRAvl9MSE7BlI4eXMQYrB9EAHN6cIrzsKH7PhYiGGsPsTnW+0NDYe3+wEPZFFvooUuScZxxLE4CTGkIhh6PPzEYpr7laCqVMUBBmyh/A4zlFE+LHDpQW26zd0mSKvsoQJCGFqF2ETfVRLrO6yrLykxh5kKmDTGd1wf7S4bwWMViwjXGX9VLEPw8K3LjqNlEj+mAZKcyA6Py93Bif4W/lkIC0bx9JHf/NiP0px0RSCrwFjsd/9bBu07yR1vGZYq1VnE9C+KUbal8zTfh5z9DbcKsVloQY2o/k9LEkiMeb6eHkFifEYhCitq3ACyTz03vk8/WjqkPm157X/xFX/BejGxU+gx89XXoH0iRqlgBpJILlpihXmfbfSQF45DO4ZgA6wuhO+g6PXUABGUBbDb9QT2RWkhaG2nYg/aUUGjMBgsgtQejMwby7nSiKqR8ckr0VMJwIHsUl1H4B8sgMq4AhL+UFgCKj19QTZ4+ULVZBJjM9wFKPOkM/Z0UstyBi/jsmULPNdzqbhiFVV+6KWKAMw= X-Forefront-PRVS: 0288CD37D9 X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10019020)(4630300001)(6009001)(39450400003)(39840400002)(39850400002)(39410400002)(39400400002)(24454002)(377454003)(36756003)(25786009)(305945005)(42186005)(7736002)(53936002)(31686004)(38730400002)(229853002)(189998001)(230700001)(4326008)(64126003)(83506001)(2906002)(6116002)(4001350100001)(53546009)(6666003)(2950100002)(50466002)(50986999)(5660300001)(54906002)(6246003)(47776003)(93886004)(81166006)(8676002)(31696002)(23746002)(65826007)(76176999)(54356999)(86362001)(6486002)(8666007)(33646002)(65806001)(42262002);DIR:OUT;SFP:1102;SCL:1;SRVR:DM5PR15MB1660;H:[IPv6:2620:10d:c0a1:1110::1048];FPR:;SPF:None;MLV:sfv;LANG:en; X-Microsoft-Exchange-Diagnostics: =?Windows-1252?Q?1;DM5PR15MB1660;23:MlYVCJEACmw4oAuSEvs611gg8NcGgmn4MDCnG?= =?Windows-1252?Q?z5dTatmHELS4CQ+SWUdRW7UGk2jHChg0k17kGo52gRoTbSLZWSHYoHmt?= =?Windows-1252?Q?3Ws75PEiSEF+2nN284TSEHTW4n/L5E/w2kZi0Cxy8pM1oxlLU73hxAfH?= =?Windows-1252?Q?ROffhc9of/CB+Xamq3TJdnz8Cgu0RIB3XIAcTC2XGvuWREQcg4nH7POd?= =?Windows-1252?Q?8JvkIuxi1Wiucu6jJds6ScCEtUvTh+FtvqHpEdyppbPYI+RdW/f/oZ8s?= =?Windows-1252?Q?nxSDN9toQpAGrK9bW0UpTJrWWrtahAWRwUYWAGLbv2+L3q8hfnybgqI/?= =?Windows-1252?Q?sIysM9IBrQSbsmtaZSUo7iRapXzTy9owIjkU0dbwSokaWn1R+WKEhIa3?= =?Windows-1252?Q?1AdKZi+aQIDDOXAfWQgvM1sXMxuCB8sqGGXf14URmjlotVeNCadTT7r6?= =?Windows-1252?Q?aj9tfcWk1gZ/6X7qQEotklHar0RLl4wAoyQGhDj4CYT279muAlS0tcd5?= =?Windows-1252?Q?bL8GJwa38HPHaCNm62+H8OKwCSkN2EC9qGJyyXWudTwFBLuF1vMMurIu?= =?Windows-1252?Q?8fhpeGqgQPQTcK/MMp2LJw0QCXnHo4USRa39mZnMGCu3qs8LgfJUBb+d?= =?Windows-1252?Q?Xd4aajc/sHOZuDY7majzGer95TPbZiW0Ne9iK57vTvFtKFSHlYvJyIIJ?= =?Windows-1252?Q?kp6+W3fVAHgHaapeR6F6OY0qtNu49hZIaJNl843Qh4+DYnHT3ry35il3?= =?Windows-1252?Q?73JRtEQRdGpt/EUWvPIfoS2zYCT4WbuKLyDR0EfbbHE1dzRWEpqwjPnC?= =?Windows-1252?Q?M8Lr5RP/cJv1VwDFZZ1YlhZb3LwYbpNiJ8mMpp5roybmPA+JNYYtaTHo?= =?Windows-1252?Q?4ORdAzYJP7IVQtLNa2HzocrTIXfnXR7IDPJvVfl0S3v0/g2FNdhqKy7W?= =?Windows-1252?Q?3WRXNwhOMzEwLFHykT9c8PDysLbO46UYcQ/CWGhWEiEMormG+oY8rwX4?= =?Windows-1252?Q?k/PTUf3dYLkqsYWlmftyjL9N6nuHM+Nk/RrwLJyswfjDvGNlq4Q0CYZ3?= =?Windows-1252?Q?tRot97D93dMTE5Js9EE9086UKff7snmK8b6MTOYK15W+3jmTdLrA9+ya?= =?Windows-1252?Q?JUPWp9nGVCsKppUpznkFS3e1puvXpFe+8RhqxUh0oe0QykPQbdWankUH?= =?Windows-1252?Q?bjzN/rdUiqBM0WzmZs8WQGLMIPpmthOaEO7JbOn6aRp4AFklOdkDn9ft?= =?Windows-1252?Q?6gCft7AnpK8zvix0mcPcqRmayIdvS0ByP+f71Lr17o5fHCQ1D3p4piOp?= =?Windows-1252?Q?2YfSm0Kfjx0hAUpGngrDvipHQ=3D=3D?= X-Microsoft-Exchange-Diagnostics: 1;DM5PR15MB1660;6:OnQTgLGVUBxgw35UA2MGea00+Udf3o2HM1sYt4UKjgWghyX0r3uQBIPrWNpZAMuhlvFLHvPO7ehH5I9rOowHg2gqA6R2uAFmU7D+JACMT1C3b6Nd9dPyp3BXgg+BAiSPQcOY2imoo9B0/4sc2TSev57HnJn3r+ogeq4IjeG0WbEoPffqE/Y/wd7I2pobNts1Yg0fA8Qgx35uLk/9xoIKdp5MkqL7pLNAkUs7NqMmiTHZQ/hWCYfB4fku6BmA/X2dJ2SL2/6FQh5Q++qr6BA1RxNFQZQpbx7SxsLfp/vC3p/46Ydijb7OaUTKJiCApSeIBM8yrnmydCy+jqy0YI5pGEFlRIfcNM4CbpqKHaBpsrWMRuPWhX6NmprlJmi0e8/G2RjLb0nO8hmeQqkXrC4OzHJyAy1hskIlwwU57YfsNDpMq3MKyE5nCXSsukcrS/djXVFryZsni69diONHmG8EhxVBrDOCNUAIFxumpvUbAeYuxVxAoB6x50zQCgDzbSIYbzDB+fU8WHHwjxaB4DSVOg==;5:L+obIMjn9RM7FOM2/1E7WPTwo1Vu/3mXyYGN9FflH87Jyg4uQ7SkVX00kAPYUw2hOZb70762f149HVtV5xwKBZQdvLhen3i4fNV5zzcvQtc9tRGZoTsmVgiW+MIRy32y3bNQ+tO5+iHoYKv5yBU2/yBhJWbynvav1Mwbx51cjNI=;24:J56FjCehdbUCc1fe675aULMJvtyShYbfM4jU0LnZ3bguj2oQ/KwupEmpRJ9z2uvOIyWEw3uCEbQwRRKzzevsRayG5I1o+46DzNuB3gDpYic= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1;DM5PR15MB1660;7:3hNeR9we5BUUe/aTU58mQ2LESQ6fZu0uqO7symLLRRDCwciRKc3J8jM6YUZijQF1Dm1kUXy4bliluJAYlOZzC8nCcJLLbhudLcqzZUTydZ55I5UOTKBpeovwFtxk5MQ6tKgsSUPJcfXk2WnPe9R0AfqaeLxLsApRe14ELyl6zFf4qBTHSlAx6lFJbm9slEr+ngX8rqtVgnlO8eTIAhHeYKdgrmUlFXsDiWw9jyhw7o5be+edu0xqLOE8nIvfnai6gXuX4Xn20hOKwZPwKU0uWqXG03zAeICgmSOzwDygDgvqg020WGFMWbyZlGcWtd0SN1clPR41OclYed7WDbKmhA==;20:ki6/qIbzbHOWtvHK49kKyXK1jFxZ9lZh/33iWflIuJ8rrORCCqStADyaxUy0V9yyTr+lz1ql9ASFAm2v9s5DYdLiEYMmjqZzY7cfN9HRkmS0oNsepcXXjEXhc2x/PDJU5peX9yXe/zHTiBHzaB2WJSKdYaOMGDO3XUMPoyoIsB8= X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Apr 2017 21:15:35.9351 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR15MB1660 X-OriginatorOrg: fb.com X-Proofpoint-Spam-Reason: safe X-FB-Internal: Safe X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2017-04-25_15:,, signatures=0 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 04/25/2017 04:49 PM, Tejun Heo wrote: > On Tue, Apr 25, 2017 at 11:49:41AM -0700, Tejun Heo wrote: >> Will try that too. I can't see why HT would change it because I see >> single CPU queues misevaluated. Just in case, you need to tune the >> test params so that it doesn't load the machine too much and that >> there are some non-CPU intensive workloads going on to purturb things >> a bit. Anyways, I'm gonna try disabling HT. > > It's finickier but after changing the duty cycle a bit, it reproduces > w/ HT off. I think the trick is setting the number of threads to the > number of logical CPUs and tune -s/-c so that p99 starts climbing up. > The following is from the root cgroup. Since it's only measuring wakeup latency, schbench is best at exposing problems when the machine is just barely below saturated. At saturation, everyone has to wait for the CPUs, and if we're relatively idle there's always a CPU to be found There's schbench -a to try and find this magic tipping point, but I haven't found a great way to automate for every kind of machine yet (sorry). -chris