From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3C047C433ED for ; Mon, 3 May 2021 11:35:28 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id F1CF5611CB for ; Mon, 3 May 2021 11:35:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233069AbhECLgS (ORCPT ); Mon, 3 May 2021 07:36:18 -0400 Received: from frasgout.his.huawei.com ([185.176.79.56]:2985 "EHLO frasgout.his.huawei.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231523AbhECLgR (ORCPT ); Mon, 3 May 2021 07:36:17 -0400 Received: from fraeml744-chm.china.huawei.com (unknown [172.18.147.226]) by frasgout.his.huawei.com (SkyGuard) with ESMTP id 4FYgjr62kBz6wlP8; Mon, 3 May 2021 19:29:36 +0800 (CST) Received: from lhreml712-chm.china.huawei.com (10.201.108.63) by fraeml744-chm.china.huawei.com (10.206.15.225) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2176.2; Mon, 3 May 2021 13:35:22 +0200 Received: from dggemi761-chm.china.huawei.com (10.1.198.147) by lhreml712-chm.china.huawei.com (10.201.108.63) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.1.2176.2; Mon, 3 May 2021 12:35:20 +0100 Received: from dggemi761-chm.china.huawei.com ([10.9.49.202]) by dggemi761-chm.china.huawei.com ([10.9.49.202]) with mapi id 15.01.2176.012; Mon, 3 May 2021 19:35:18 +0800 From: "Song Bao Hua (Barry Song)" To: Dietmar Eggemann , Vincent Guittot CC: "tim.c.chen@linux.intel.com" , "catalin.marinas@arm.com" , "will@kernel.org" , "rjw@rjwysocki.net" , "bp@alien8.de" , "tglx@linutronix.de" , "mingo@redhat.com" , "lenb@kernel.org" , "peterz@infradead.org" , "rostedt@goodmis.org" , "bsegall@google.com" , "mgorman@suse.de" , "msys.mizuma@gmail.com" , "valentin.schneider@arm.com" , "gregkh@linuxfoundation.org" , Jonathan Cameron , "juri.lelli@redhat.com" , "mark.rutland@arm.com" , "sudeep.holla@arm.com" , "aubrey.li@linux.intel.com" , "linux-arm-kernel@lists.infradead.org" , "linux-kernel@vger.kernel.org" , "linux-acpi@vger.kernel.org" , "x86@kernel.org" , "xuwei (O)" , "Zengtao (B)" , "guodong.xu@linaro.org" , yangyicong , "Liguozhu (Kenneth)" , "linuxarm@openeuler.org" , "hpa@zytor.com" Subject: RE: [RFC PATCH v6 3/4] scheduler: scan idle cpu in cluster for tasks within one LLC Thread-Topic: [RFC PATCH v6 3/4] scheduler: scan idle cpu in cluster for tasks within one LLC Thread-Index: AQHXPa2htgkN1X7dCEatlQZ1LQ756arRBrBQgACVCdA= Date: Mon, 3 May 2021 11:35:18 +0000 Message-ID: <4d1f063504b1420c9f836d1f1a7f8e77@hisilicon.com> References: <20210420001844.9116-1-song.bao.hua@hisilicon.com> <20210420001844.9116-4-song.bao.hua@hisilicon.com> <80f489f9-8c88-95d8-8241-f0cfd2c2ac66@arm.com> <8b5277d9-e367-566d-6bd1-44ac78d21d3f@arm.com> <185746c4d02a485ca8f3509439328b26@hisilicon.com> Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.126.203.132] Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 MIME-Version: 1.0 X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-acpi@vger.kernel.org DQoNCj4gLS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCj4gRnJvbTogU29uZyBCYW8gSHVhIChC YXJyeSBTb25nKQ0KPiBTZW50OiBNb25kYXksIE1heSAzLCAyMDIxIDY6MTIgUE0NCj4gVG86ICdE aWV0bWFyIEVnZ2VtYW5uJyA8ZGlldG1hci5lZ2dlbWFubkBhcm0uY29tPjsgVmluY2VudCBHdWl0 dG90DQo+IDx2aW5jZW50Lmd1aXR0b3RAbGluYXJvLm9yZz4NCj4gQ2M6IHRpbS5jLmNoZW5AbGlu dXguaW50ZWwuY29tOyBjYXRhbGluLm1hcmluYXNAYXJtLmNvbTsgd2lsbEBrZXJuZWwub3JnOw0K PiByandAcmp3eXNvY2tpLm5ldDsgYnBAYWxpZW44LmRlOyB0Z2x4QGxpbnV0cm9uaXguZGU7IG1p bmdvQHJlZGhhdC5jb207DQo+IGxlbmJAa2VybmVsLm9yZzsgcGV0ZXJ6QGluZnJhZGVhZC5vcmc7 IHJvc3RlZHRAZ29vZG1pcy5vcmc7DQo+IGJzZWdhbGxAZ29vZ2xlLmNvbTsgbWdvcm1hbkBzdXNl LmRlOyBtc3lzLm1penVtYUBnbWFpbC5jb207DQo+IHZhbGVudGluLnNjaG5laWRlckBhcm0uY29t OyBncmVna2hAbGludXhmb3VuZGF0aW9uLm9yZzsgSm9uYXRoYW4gQ2FtZXJvbg0KPiA8am9uYXRo YW4uY2FtZXJvbkBodWF3ZWkuY29tPjsganVyaS5sZWxsaUByZWRoYXQuY29tOyBtYXJrLnJ1dGxh bmRAYXJtLmNvbTsNCj4gc3VkZWVwLmhvbGxhQGFybS5jb207IGF1YnJleS5saUBsaW51eC5pbnRl bC5jb207DQo+IGxpbnV4LWFybS1rZXJuZWxAbGlzdHMuaW5mcmFkZWFkLm9yZzsgbGludXgta2Vy bmVsQHZnZXIua2VybmVsLm9yZzsNCj4gbGludXgtYWNwaUB2Z2VyLmtlcm5lbC5vcmc7IHg4NkBr ZXJuZWwub3JnOyB4dXdlaSAoTykgPHh1d2VpNUBodWF3ZWkuY29tPjsNCj4gWmVuZ3RhbyAoQikg PHByaW1lLnplbmdAaGlzaWxpY29uLmNvbT47IGd1b2RvbmcueHVAbGluYXJvLm9yZzsgeWFuZ3lp Y29uZw0KPiA8eWFuZ3lpY29uZ0BodWF3ZWkuY29tPjsgTGlndW96aHUgKEtlbm5ldGgpIDxsaWd1 b3podUBoaXNpbGljb24uY29tPjsNCj4gbGludXhhcm1Ab3BlbmV1bGVyLm9yZzsgaHBhQHp5dG9y LmNvbQ0KPiBTdWJqZWN0OiBSRTogW1JGQyBQQVRDSCB2NiAzLzRdIHNjaGVkdWxlcjogc2NhbiBp ZGxlIGNwdSBpbiBjbHVzdGVyIGZvciB0YXNrcw0KPiB3aXRoaW4gb25lIExMQw0KPiANCj4gDQo+ IA0KPiA+IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQo+ID4gRnJvbTogRGlldG1hciBFZ2dl bWFubiBbbWFpbHRvOmRpZXRtYXIuZWdnZW1hbm5AYXJtLmNvbV0NCj4gPiBTZW50OiBGcmlkYXks IEFwcmlsIDMwLCAyMDIxIDEwOjQzIFBNDQo+ID4gVG86IFNvbmcgQmFvIEh1YSAoQmFycnkgU29u ZykgPHNvbmcuYmFvLmh1YUBoaXNpbGljb24uY29tPjsgVmluY2VudCBHdWl0dG90DQo+ID4gPHZp bmNlbnQuZ3VpdHRvdEBsaW5hcm8ub3JnPg0KPiA+IENjOiB0aW0uYy5jaGVuQGxpbnV4LmludGVs LmNvbTsgY2F0YWxpbi5tYXJpbmFzQGFybS5jb207IHdpbGxAa2VybmVsLm9yZzsNCj4gPiByandA cmp3eXNvY2tpLm5ldDsgYnBAYWxpZW44LmRlOyB0Z2x4QGxpbnV0cm9uaXguZGU7IG1pbmdvQHJl ZGhhdC5jb207DQo+ID4gbGVuYkBrZXJuZWwub3JnOyBwZXRlcnpAaW5mcmFkZWFkLm9yZzsgcm9z dGVkdEBnb29kbWlzLm9yZzsNCj4gPiBic2VnYWxsQGdvb2dsZS5jb207IG1nb3JtYW5Ac3VzZS5k ZTsgbXN5cy5taXp1bWFAZ21haWwuY29tOw0KPiA+IHZhbGVudGluLnNjaG5laWRlckBhcm0uY29t OyBncmVna2hAbGludXhmb3VuZGF0aW9uLm9yZzsgSm9uYXRoYW4gQ2FtZXJvbg0KPiA+IDxqb25h dGhhbi5jYW1lcm9uQGh1YXdlaS5jb20+OyBqdXJpLmxlbGxpQHJlZGhhdC5jb207DQo+IG1hcmsu cnV0bGFuZEBhcm0uY29tOw0KPiA+IHN1ZGVlcC5ob2xsYUBhcm0uY29tOyBhdWJyZXkubGlAbGlu dXguaW50ZWwuY29tOw0KPiA+IGxpbnV4LWFybS1rZXJuZWxAbGlzdHMuaW5mcmFkZWFkLm9yZzsg bGludXgta2VybmVsQHZnZXIua2VybmVsLm9yZzsNCj4gPiBsaW51eC1hY3BpQHZnZXIua2VybmVs Lm9yZzsgeDg2QGtlcm5lbC5vcmc7IHh1d2VpIChPKSA8eHV3ZWk1QGh1YXdlaS5jb20+Ow0KPiA+ IFplbmd0YW8gKEIpIDxwcmltZS56ZW5nQGhpc2lsaWNvbi5jb20+OyBndW9kb25nLnh1QGxpbmFy by5vcmc7IHlhbmd5aWNvbmcNCj4gPiA8eWFuZ3lpY29uZ0BodWF3ZWkuY29tPjsgTGlndW96aHUg KEtlbm5ldGgpIDxsaWd1b3podUBoaXNpbGljb24uY29tPjsNCj4gPiBsaW51eGFybUBvcGVuZXVs ZXIub3JnOyBocGFAenl0b3IuY29tDQo+ID4gU3ViamVjdDogUmU6IFtSRkMgUEFUQ0ggdjYgMy80 XSBzY2hlZHVsZXI6IHNjYW4gaWRsZSBjcHUgaW4gY2x1c3RlciBmb3IgdGFza3MNCj4gPiB3aXRo aW4gb25lIExMQw0KPiA+DQo+ID4gT24gMjkvMDQvMjAyMSAwMDo0MSwgU29uZyBCYW8gSHVhIChC YXJyeSBTb25nKSB3cm90ZToNCj4gPiA+DQo+ID4gPg0KPiA+ID4+IC0tLS0tT3JpZ2luYWwgTWVz c2FnZS0tLS0tDQo+ID4gPj4gRnJvbTogRGlldG1hciBFZ2dlbWFubiBbbWFpbHRvOmRpZXRtYXIu ZWdnZW1hbm5AYXJtLmNvbV0NCj4gPg0KPiA+IFsuLi5dDQo+ID4NCj4gPiA+Pj4+PiBGcm9tOiBE aWV0bWFyIEVnZ2VtYW5uIFttYWlsdG86ZGlldG1hci5lZ2dlbWFubkBhcm0uY29tXQ0KPiA+ID4+ DQo+ID4gPj4gWy4uLl0NCj4gPiA+Pg0KPiA+ID4+Pj4+IE9uIDIwLzA0LzIwMjEgMDI6MTgsIEJh cnJ5IFNvbmcgd3JvdGU6DQo+ID4NCj4gPiBbLi4uXQ0KPiA+DQo+ID4gPiBUaG91Z2ggd2Ugd2ls bCBuZXZlciBnbyB0byBzbG93IHBhdGgsIHdha2Vfd2lkZSgpIHdpbGwgYWZmZWN0IHdhbnRfYWZm aW5lLA0KPiA+ID4gc28gZXZlbnR1YWxseSBhZmZlY3QgdGhlICJuZXdfY3B1Ij8NCj4gPg0KPiA+ IHllcy4NCj4gPg0KPiA+ID4NCj4gPiA+IAlmb3JfZWFjaF9kb21haW4oY3B1LCB0bXApIHsNCj4g PiA+IAkJLyoNCj4gPiA+IAkJICogSWYgYm90aCAnY3B1JyBhbmQgJ3ByZXZfY3B1JyBhcmUgcGFy dCBvZiB0aGlzIGRvbWFpbiwNCj4gPiA+IAkJICogY3B1IGlzIGEgdmFsaWQgU0RfV0FLRV9BRkZJ TkUgdGFyZ2V0Lg0KPiA+ID4gCQkgKi8NCj4gPiA+IAkJaWYgKHdhbnRfYWZmaW5lICYmICh0bXAt PmZsYWdzICYgU0RfV0FLRV9BRkZJTkUpICYmDQo+ID4gPiAJCSAgICBjcHVtYXNrX3Rlc3RfY3B1 KHByZXZfY3B1LCBzY2hlZF9kb21haW5fc3Bhbih0bXApKSkgew0KPiA+ID4gCQkJaWYgKGNwdSAh PSBwcmV2X2NwdSkNCj4gPiA+IAkJCQluZXdfY3B1ID0gd2FrZV9hZmZpbmUodG1wLCBwLCBjcHUs IHByZXZfY3B1LCBzeW5jKTsNCj4gPiA+DQo+ID4gPiAJCQlzZCA9IE5VTEw7IC8qIFByZWZlciB3 YWtlX2FmZmluZSBvdmVyIGJhbGFuY2UgZmxhZ3MgKi8NCj4gPiA+IAkJCWJyZWFrOw0KPiA+ID4g CQl9DQo+ID4gPg0KPiA+ID4gCQlpZiAodG1wLT5mbGFncyAmIHNkX2ZsYWcpDQo+ID4gPiAJCQlz ZCA9IHRtcDsNCj4gPiA+IAkJZWxzZSBpZiAoIXdhbnRfYWZmaW5lKQ0KPiA+ID4gCQkJYnJlYWs7 DQo+ID4gPiAJfQ0KPiA+ID4NCj4gPiA+IElmIHdha2VfYWZmaW5lIGlzIGZhbHNlLCB0aGUgYWJv dmUgd29uJ3QgZXhlY3V0ZSwgbmV3X2NwdSh0YXJnZXQpIHdpbGwNCj4gPiA+IGFsd2F5cyBiZSAi cHJldl9jcHUiPyBzbyB3aGVuIHRhc2sgc2l6ZSA+IGNsdXN0ZXIgc2l6ZSBpbiB3YWtlX3dpZGUo KSwNCj4gPiA+IHRoaXMgbWVhbnMgd2Ugd29uJ3QgcHVsbCB0aGUgd2FrZWUgdG8gdGhlIGNsdXN0 ZXIgb2Ygd2FrZXI/IEl0IHNlZW1zDQo+ID4gPiBzZW5zaWJsZS4NCj4gPg0KPiA+IFdoYXQgaXMg YHRhc2sgc2l6ZWAgaGVyZT8NCj4gPg0KPiA+IFRoZSBjcml0ZXJpb24gaXMgYCEoc2xhdmUgPCBm YWN0b3IgfHwgbWFzdGVyIDwgc2xhdmUgKiBmYWN0b3IpYCBvcg0KPiA+IGBzbGF2ZSA+PSBmYWN0 b3IgJiYgbWFzdGVyID49IHNsYXZlICogZmFjdG9yYCB0byB3YWtlIHdpZGUuDQo+ID4NCj4gDQo+ IFllcy4gRm9yICJ0YXNrIHNpemUiLCBJIGFjdHVhbGx5IG1lYW4gYSBidW5kbGUgb2Ygd2FrZXIt d2FrZWUgdGFza3MNCj4gd2hpY2ggY2FuIG1ha2UgInNsYXZlID49IGZhY3RvciAmJiBtYXN0ZXIg Pj0gc2xhdmUgKiBmYWN0b3IiIGVpdGhlcg0KPiB0cnVlIG9yIGZhbHNlLCB0aGVuIGNoYW5nZSB0 aGUgdGFyZ2V0IGNwdSB3aGVyZSB3ZSBhcmUgZ29pbmcgdG8gc2Nhbg0KPiBmcm9tLg0KPiBOb3cg c2luY2UgSSBoYXZlIG1vdmVkIHRvIGNsdXN0ZXIgbGV2ZWwgd2hlbiB0YXNrcyBoYXZlIGJlZW4g aW4gc2FtZQ0KPiBMTEMgbGV2ZWwsIGl0IHNlZW1zIGl0IHdvdWxkIGJlIG1vcmUgc2Vuc2libGUg dG8gdXNlICJjbHVzdGVyX3NpemUiIGFzDQo+IGZhY3Rvcj8NCj4gDQo+ID4gSSBzZWUgdGhhdCBz aW5jZSB5b3UgZWZmZWN0aXZlbHkgY2hhbmdlIHRoZSBzY2hlZCBkb21haW4gc2l6ZSBmcm9tIExM Qw0KPiA+IHRvIENMVVNURVIgKGUuZy4gMjQtPjYpIGZvciB3YWtldXBzIHdpdGggY3B1IGFuZCBw cmV2X2NwdSBzaGFyaW5nIExMQw0KPiA+IChoZW5jZSB0aGUgYG51bWFjdGwgLU4gMGAgaW4geW91 ciB3b3JrbG9hZCksIHdha2Vfd2lkZSgpIGhhcyB0byB0YWtlDQo+ID4gQ0xVU1RFUiBzaXplIGlu dG8gY29uc2lkZXJhdGlvbi4NCj4gPg0KPiA+IEkgd2FzIHdvbmRlcmluZyBpZiB5b3Ugc2F3IHdh a2Vfd2lkZSgpIHJldHVybmluZyAxIHdpdGggeW91ciB1c2UgY2FzZXM6DQo+ID4NCj4gPiBudW1h Y3RsIC1OIDAgL3Vzci9saWIvbG1iZW5jaC9iaW4vc3RyZWFtIC1QIFs2LDEyXSAtTSAxMDI0TSAt TiA1DQo+IA0KPiBJIGNvdWxkbid0IG1ha2Ugd2FrZV93aWRlIHJldHVybiAxIGJ5IHRoZSBhYm92 ZSBzdHJlYW0gY29tbWFuZC4NCj4gQW5kIEkgY2FuJ3QgcmVwcm9kdWNlIGl0IGJ5IGEgMToxKG1v bm9nYW1vdXMpIGhhY2tiZW5jaCAiLWYgMSIuDQo+IA0KPiBCdXQgSSBhbSBhYmxlIHRvIHJlcHJv ZHVjZSB0aGlzIGlzc3VlIGJ5IGEgTTpOIGhhY2tiZW5jaCwgZm9yIGV4YW1wbGU6DQo+IA0KPiBu dW1hY3RsIC1OIDAgaGFja2JlbmNoIC1wIC1UIC1mIDEwIC1sIDIwMDAwIC1nIDENCj4gDQo+IGhh Y2tiZW5jaCB3aWxsIGNyZWF0ZSAxMCBzZW5kZXJzIHdoaWNoIHdpbGwgc2VuZCBtZXNzYWdlcyB0 byAxMA0KPiByZWNlaXZlcnMuIChFYWNoIHNlbmRlciBjYW4gc2VuZCBtZXNzYWdlcyB0byBhbGwg MTAgcmVjZWl2ZXJzLikNCj4gDQo+IEkndmUgb2Z0ZW4gc2VlbiBmbGlwcyBsaWtlOg0KPiB3YWtl ciB3YWtlZQ0KPiAxNTAxICAzOQ0KPiAxNTA5ICAxNw0KPiAxMSAgIDEzMjANCj4gMTMgICAyMDE2 DQo+IA0KPiAxMSwgMTMsIDE3IGlzIHNtYWxsZXIgdGhhbiBMTEMgYnV0IGxhcmdlciB0aGFuIGNs dXN0ZXIuIFNvIHRoZSB3YWtlX3dpZGUoKQ0KPiB1c2luZyBjbHVzdGVyIGZhY3RvciB3aWxsIHJl dHVybiAxLCBvbiB0aGUgb3RoZXIgaGFuZCwgaWYgd2UgYWx3YXlzIHVzZQ0KPiBsbGNfc2l6ZSBh cyBmYWN0b3IsIGl0IHdpbGwgcmV0dXJuIDAuDQo+IA0KPiBIb3dldmVyLCBpdCBzZWVtcyB0aGUg Y2hhbmdlIGluIHdha2Vfd2lkZSgpIGNvdWxkIGJyaW5nIHNvbWUgbmVnYXRpdmUNCj4gaW5mbHVl bmNlIHRvIE06TiByZWxhdGlvbnNoaXAoLWYgMTApIGFjY29yZGluZyB0byB0ZXN0cyBtYWRlIHRv ZGF5IGJ5Og0KPiANCj4gbnVtYWN0bCAtTiAwIGhhY2tiZW5jaCAtcCAtVCAtZiAxMCAtbCAyMDAw MCAtZyAkMQ0KPiANCj4gZyAgICAgICAgICAgICA9ICAgICAgMSAgICAgMiAgICAgICAzICAgICAg IDQNCj4gY2x1c3Rlcl9zaXplICAgICAwLjU3NjggMC42NTc4ICAwLjgxMTcgMS4wMTE5DQo+IExM Q19zaXplICAgICAgICAgMC41NDc5IDAuNjE2MiAgMC42OTIyIDAuNzc1NA0KPiANCj4gQWx3YXlz IHVzaW5nIGxsY19zaXplIGFzIGZhY3RvciBpbiB3YWtlX3dpZGUgc3RpbGwgc2hvd3MgYmV0dGVy IHJlc3VsdA0KPiBpbiB0aGUgMTA6MTAgcG9seWdhbW91cyBoYWNrYmVuY2guDQo+IA0KPiBTbyBp dCBzZWVtcyB0aGUgYHNsYXZlID49IGZhY3RvciAmJiBtYXN0ZXIgPj0gc2xhdmUgKiBmYWN0b3Jg IGlzbid0DQo+IGEgc3VpdGFibGUgY3JpdGVyaW9uIGZvciBjbHVzdGVyIHNpemU/DQoNCk9uIHRo ZSBvdGhlciBoYW5kLCBhY2NvcmRpbmcgdG8gInNjaGVkOiBJbXBsZW1lbnQgc21hcnRlciB3YWtl LWFmZmluZSBsb2dpYyINCmh0dHBzOi8vZ2l0Lmtlcm5lbC5vcmcvcHViL3NjbS9saW51eC9rZXJu ZWwvZ2l0L3RvcnZhbGRzL2xpbnV4LmdpdC9jb21taXQvP2lkPTYyNDcwNDE5DQoNClByb3BlciBm YWN0b3IgaW4gd2FrZV93aWRlIGlzIG1haW5seSBiZW5lZmljaWFsIG9mIDE6biB0YXNrcyBsaWtl IHBvc3RncmVzcWwvcGdiZW5jaC4NClNvIHVzaW5nIHRoZSBzbWFsbGVyIGNsdXN0ZXIgc2l6ZSBh cyBmYWN0b3IgbWlnaHQgaGVscCBtYWtlIHdha2VfYWZmaW5lIGZhbHNlIHNvDQppbXByb3ZlIHBn YmVuY2guDQoNCkZyb20gdGhlIGNvbW1pdCBsb2csIHdoaWxlIGNsaWVudHMgPSAgMipjcHVzLCB0 aGUgY29tbWl0IG1hZGUgdGhlIGJpZ2dlc3QNCmltcHJvdmVtZW50LiBJbiBteSBjYXNlLCBJdCBz aG91bGQgYmUgY2xpZW50cz00OCBmb3IgYSBtYWNoaW5lIHdob3NlIExMQw0Kc2l6ZSBpcyAyNC4N Cg0KSW4gTGludXgsIEkgY3JlYXRlZCBhIDI0ME1CIGRhdGFiYXNlIGFuZCByYW4gInBnYmVuY2gg LWMgNDggLVMgLVQgMjAgcGdiZW5jaCINCnVuZGVyIHR3byBkaWZmZXJlbnQgc2NlbmFyaW9zOg0K MS4gcGFnZSBjYWNoZSBhbHdheXMgaGl0LCBzbyBubyByZWFsIEkvTyBmb3IgZGF0YWJhc2UgcmVh ZA0KMi4gZWNobyAzID4gL3Byb2Mvc3lzL3ZtL2Ryb3BfY2FjaGVzDQoNCkZvciBjYXNlIDEsIHVz aW5nIGNsdXN0ZXJfc2l6ZSBhbmQgdXNpbmcgbGxjX3NpemUgd2lsbCByZXN1bHQgaW4gc2ltaWxh cg0KdHBzPSB+MTA4MDAwLCBhbGwgb2YgMjQgY3B1cyBoYXZlIDEwMCUgY3B1IHV0aWxpemF0aW9u Lg0KDQpGb3IgY2FzZSAyLCB1c2luZyBsbGNfc2l6ZSBzdGlsbCBzaG93cyBiZXR0ZXIgcGVyZm9y bWFuY2UuDQoNCnRwcyBmb3IgZWFjaCB0ZXN0IHJvdW5kKGNsdXN0ZXIgc2l6ZSBhcyBmYWN0b3Ig aW4gd2FrZV93aWRlKToNCjEzOTguNDUwODg3IDEyNzUuMDIwNDAxIDE2MzIuNTQyNDM3IDE0MTIu MjQxNjI3IDE2MTEuMDk1NjkyIDEzODEuMzU0Mjk0IDE1MzkuODc3MTQ2DQphdmcgdHBzID0gMTQ2 NA0KDQp0cHMgZm9yIGVhY2ggdGVzdCByb3VuZChsbGMgc2l6ZSBhcyBmYWN0b3IgaW4gd2FrZV93 aWRlKToNCjE3MTguNDAyOTgzIDE0NDMuMTY5ODIzIDE1MDIuMzUzODIzIDE2MDcuNDE1ODYxIDE1 OTcuMzk2OTI0IDE3NDUuNjUxODE0IDE4NzYuODAyMTY4DQphdmcgdHBzID0gMTY0MSAgKCsxMiUp DQoNCnNvIGl0IHNlZW1zIHVzaW5nIGNsdXN0ZXJfc2l6ZSBhcyBmYWN0b3IgaW4gInNsYXZlID49 IGZhY3RvciAmJiBtYXN0ZXIgPj0gc2xhdmUgKg0KZmFjdG9yIiBpc24ndCBhIGdvb2QgY2hvaWNl IGZvciBteSBtYWNoaW5lIGF0IGxlYXN0Lg0KDQpUaGFua3MNCkJhcnJ5DQo= From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, MENTIONS_GIT_HOSTING,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5D5B0C433ED for ; Mon, 3 May 2021 11:38:48 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7A3E061244 for ; Mon, 3 May 2021 11:38:47 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7A3E061244 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=hisilicon.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:Message-ID:Date:Subject:CC: To:From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:List-Owner; bh=LsgC+txvvnKP+i5LqUek+RXq93+uIkNd52k0WaF89m8=; b=c9FvZjfEuSuczhdz2Gt35Kkw/ 4FH0RGcEfGk0fOTFGWEcuKF5WIj5qVg/3DdlSQrkmzlqklVwZosKmWsf25CbilDq5j5a6qxvEkm07 fq0NgP+VWFXI1a0lgz6uMNtSpSm8Jw8oaFsZEl8a/VTTxOnlcfLKLwPVRziFpTChGU88E5WHYR48a IQ3Peg7AHdgEQLvYxN73KjATYbxWupe+ao9vAkDR1clApdRo61XBW//S2s2PszEaYB2KXHrWBrnJJ +79Q1bwvO02VtS/VryLg/c1f0V7rMtR3Q5QNVtHbmXh5/bkpzUMSwLO+VaO7McglX/NX/5DW64Y/H jBVRl9+Cg==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1ldWry-00DlL3-BL; Mon, 03 May 2021 11:36:12 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1ldWrM-00DlIv-MR for linux-arm-kernel@desiato.infradead.org; Mon, 03 May 2021 11:35:35 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=MIME-Version: Content-Transfer-Encoding:Content-Type:References:Message-ID:Date:Subject:CC: To:From:Sender:Reply-To:Content-ID:Content-Description:In-Reply-To; bh=u4RK0ibjcEnkY3bUOKdqruG3Gn+F/C/dKyApK1ekQGY=; b=Y/wljKiQQg/Xgt2fZwbugifgeO 4eJA5ZoIAcTgWfvUWFYHGKnp79x1GzlfwYgH6KEYrE9pSfUCzAHAUf0VEWOGmZ89Gv4sZJ5ltqQ/Z CvZv1RAOH8ntsBpM8yxjJ2hTcXKxTLRs04adjP6WGs5loUQOKCIFUS2h+7TPtC4O5hwf5hRa55Why RU7fxDq8/qbbyNLnrO2K90Ft0aLTe+IlPudxXiafpi/NLyUa3ez2udX4d5y9xEXhhxQRorOqtqL5I 37sWLrT26w2SBkv8vqPS8QEtjyGQEm7GdrWsdd0ZczxnftwZRPVVv8uMxPuPzdGZ8octkgFWUm7xA we5WKO6A==; Received: from frasgout.his.huawei.com ([185.176.79.56]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1ldWrH-00315X-NJ for linux-arm-kernel@lists.infradead.org; Mon, 03 May 2021 11:35:30 +0000 Received: from fraeml744-chm.china.huawei.com (unknown [172.18.147.226]) by frasgout.his.huawei.com (SkyGuard) with ESMTP id 4FYgjr62kBz6wlP8; Mon, 3 May 2021 19:29:36 +0800 (CST) Received: from lhreml712-chm.china.huawei.com (10.201.108.63) by fraeml744-chm.china.huawei.com (10.206.15.225) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2176.2; Mon, 3 May 2021 13:35:22 +0200 Received: from dggemi761-chm.china.huawei.com (10.1.198.147) by lhreml712-chm.china.huawei.com (10.201.108.63) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.1.2176.2; Mon, 3 May 2021 12:35:20 +0100 Received: from dggemi761-chm.china.huawei.com ([10.9.49.202]) by dggemi761-chm.china.huawei.com ([10.9.49.202]) with mapi id 15.01.2176.012; Mon, 3 May 2021 19:35:18 +0800 From: "Song Bao Hua (Barry Song)" To: Dietmar Eggemann , Vincent Guittot CC: "tim.c.chen@linux.intel.com" , "catalin.marinas@arm.com" , "will@kernel.org" , "rjw@rjwysocki.net" , "bp@alien8.de" , "tglx@linutronix.de" , "mingo@redhat.com" , "lenb@kernel.org" , "peterz@infradead.org" , "rostedt@goodmis.org" , "bsegall@google.com" , "mgorman@suse.de" , "msys.mizuma@gmail.com" , "valentin.schneider@arm.com" , "gregkh@linuxfoundation.org" , Jonathan Cameron , "juri.lelli@redhat.com" , "mark.rutland@arm.com" , "sudeep.holla@arm.com" , "aubrey.li@linux.intel.com" , "linux-arm-kernel@lists.infradead.org" , "linux-kernel@vger.kernel.org" , "linux-acpi@vger.kernel.org" , "x86@kernel.org" , "xuwei (O)" , "Zengtao (B)" , "guodong.xu@linaro.org" , yangyicong , "Liguozhu (Kenneth)" , "linuxarm@openeuler.org" , "hpa@zytor.com" Subject: RE: [RFC PATCH v6 3/4] scheduler: scan idle cpu in cluster for tasks within one LLC Thread-Topic: [RFC PATCH v6 3/4] scheduler: scan idle cpu in cluster for tasks within one LLC Thread-Index: AQHXPa2htgkN1X7dCEatlQZ1LQ756arRBrBQgACVCdA= Date: Mon, 3 May 2021 11:35:18 +0000 Message-ID: <4d1f063504b1420c9f836d1f1a7f8e77@hisilicon.com> References: <20210420001844.9116-1-song.bao.hua@hisilicon.com> <20210420001844.9116-4-song.bao.hua@hisilicon.com> <80f489f9-8c88-95d8-8241-f0cfd2c2ac66@arm.com> <8b5277d9-e367-566d-6bd1-44ac78d21d3f@arm.com> <185746c4d02a485ca8f3509439328b26@hisilicon.com> Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.126.203.132] MIME-Version: 1.0 X-CFilter-Loop: Reflected X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210503_043528_064416_8CF1E245 X-CRM114-Status: GOOD ( 34.58 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org > -----Original Message----- > From: Song Bao Hua (Barry Song) > Sent: Monday, May 3, 2021 6:12 PM > To: 'Dietmar Eggemann' ; Vincent Guittot > > Cc: tim.c.chen@linux.intel.com; catalin.marinas@arm.com; will@kernel.org; > rjw@rjwysocki.net; bp@alien8.de; tglx@linutronix.de; mingo@redhat.com; > lenb@kernel.org; peterz@infradead.org; rostedt@goodmis.org; > bsegall@google.com; mgorman@suse.de; msys.mizuma@gmail.com; > valentin.schneider@arm.com; gregkh@linuxfoundation.org; Jonathan Cameron > ; juri.lelli@redhat.com; mark.rutland@arm.com; > sudeep.holla@arm.com; aubrey.li@linux.intel.com; > linux-arm-kernel@lists.infradead.org; linux-kernel@vger.kernel.org; > linux-acpi@vger.kernel.org; x86@kernel.org; xuwei (O) ; > Zengtao (B) ; guodong.xu@linaro.org; yangyicong > ; Liguozhu (Kenneth) ; > linuxarm@openeuler.org; hpa@zytor.com > Subject: RE: [RFC PATCH v6 3/4] scheduler: scan idle cpu in cluster for tasks > within one LLC > > > > > -----Original Message----- > > From: Dietmar Eggemann [mailto:dietmar.eggemann@arm.com] > > Sent: Friday, April 30, 2021 10:43 PM > > To: Song Bao Hua (Barry Song) ; Vincent Guittot > > > > Cc: tim.c.chen@linux.intel.com; catalin.marinas@arm.com; will@kernel.org; > > rjw@rjwysocki.net; bp@alien8.de; tglx@linutronix.de; mingo@redhat.com; > > lenb@kernel.org; peterz@infradead.org; rostedt@goodmis.org; > > bsegall@google.com; mgorman@suse.de; msys.mizuma@gmail.com; > > valentin.schneider@arm.com; gregkh@linuxfoundation.org; Jonathan Cameron > > ; juri.lelli@redhat.com; > mark.rutland@arm.com; > > sudeep.holla@arm.com; aubrey.li@linux.intel.com; > > linux-arm-kernel@lists.infradead.org; linux-kernel@vger.kernel.org; > > linux-acpi@vger.kernel.org; x86@kernel.org; xuwei (O) ; > > Zengtao (B) ; guodong.xu@linaro.org; yangyicong > > ; Liguozhu (Kenneth) ; > > linuxarm@openeuler.org; hpa@zytor.com > > Subject: Re: [RFC PATCH v6 3/4] scheduler: scan idle cpu in cluster for tasks > > within one LLC > > > > On 29/04/2021 00:41, Song Bao Hua (Barry Song) wrote: > > > > > > > > >> -----Original Message----- > > >> From: Dietmar Eggemann [mailto:dietmar.eggemann@arm.com] > > > > [...] > > > > >>>>> From: Dietmar Eggemann [mailto:dietmar.eggemann@arm.com] > > >> > > >> [...] > > >> > > >>>>> On 20/04/2021 02:18, Barry Song wrote: > > > > [...] > > > > > Though we will never go to slow path, wake_wide() will affect want_affine, > > > so eventually affect the "new_cpu"? > > > > yes. > > > > > > > > for_each_domain(cpu, tmp) { > > > /* > > > * If both 'cpu' and 'prev_cpu' are part of this domain, > > > * cpu is a valid SD_WAKE_AFFINE target. > > > */ > > > if (want_affine && (tmp->flags & SD_WAKE_AFFINE) && > > > cpumask_test_cpu(prev_cpu, sched_domain_span(tmp))) { > > > if (cpu != prev_cpu) > > > new_cpu = wake_affine(tmp, p, cpu, prev_cpu, sync); > > > > > > sd = NULL; /* Prefer wake_affine over balance flags */ > > > break; > > > } > > > > > > if (tmp->flags & sd_flag) > > > sd = tmp; > > > else if (!want_affine) > > > break; > > > } > > > > > > If wake_affine is false, the above won't execute, new_cpu(target) will > > > always be "prev_cpu"? so when task size > cluster size in wake_wide(), > > > this means we won't pull the wakee to the cluster of waker? It seems > > > sensible. > > > > What is `task size` here? > > > > The criterion is `!(slave < factor || master < slave * factor)` or > > `slave >= factor && master >= slave * factor` to wake wide. > > > > Yes. For "task size", I actually mean a bundle of waker-wakee tasks > which can make "slave >= factor && master >= slave * factor" either > true or false, then change the target cpu where we are going to scan > from. > Now since I have moved to cluster level when tasks have been in same > LLC level, it seems it would be more sensible to use "cluster_size" as > factor? > > > I see that since you effectively change the sched domain size from LLC > > to CLUSTER (e.g. 24->6) for wakeups with cpu and prev_cpu sharing LLC > > (hence the `numactl -N 0` in your workload), wake_wide() has to take > > CLUSTER size into consideration. > > > > I was wondering if you saw wake_wide() returning 1 with your use cases: > > > > numactl -N 0 /usr/lib/lmbench/bin/stream -P [6,12] -M 1024M -N 5 > > I couldn't make wake_wide return 1 by the above stream command. > And I can't reproduce it by a 1:1(monogamous) hackbench "-f 1". > > But I am able to reproduce this issue by a M:N hackbench, for example: > > numactl -N 0 hackbench -p -T -f 10 -l 20000 -g 1 > > hackbench will create 10 senders which will send messages to 10 > receivers. (Each sender can send messages to all 10 receivers.) > > I've often seen flips like: > waker wakee > 1501 39 > 1509 17 > 11 1320 > 13 2016 > > 11, 13, 17 is smaller than LLC but larger than cluster. So the wake_wide() > using cluster factor will return 1, on the other hand, if we always use > llc_size as factor, it will return 0. > > However, it seems the change in wake_wide() could bring some negative > influence to M:N relationship(-f 10) according to tests made today by: > > numactl -N 0 hackbench -p -T -f 10 -l 20000 -g $1 > > g = 1 2 3 4 > cluster_size 0.5768 0.6578 0.8117 1.0119 > LLC_size 0.5479 0.6162 0.6922 0.7754 > > Always using llc_size as factor in wake_wide still shows better result > in the 10:10 polygamous hackbench. > > So it seems the `slave >= factor && master >= slave * factor` isn't > a suitable criterion for cluster size? On the other hand, according to "sched: Implement smarter wake-affine logic" https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=62470419 Proper factor in wake_wide is mainly beneficial of 1:n tasks like postgresql/pgbench. So using the smaller cluster size as factor might help make wake_affine false so improve pgbench. >From the commit log, while clients = 2*cpus, the commit made the biggest improvement. In my case, It should be clients=48 for a machine whose LLC size is 24. In Linux, I created a 240MB database and ran "pgbench -c 48 -S -T 20 pgbench" under two different scenarios: 1. page cache always hit, so no real I/O for database read 2. echo 3 > /proc/sys/vm/drop_caches For case 1, using cluster_size and using llc_size will result in similar tps= ~108000, all of 24 cpus have 100% cpu utilization. For case 2, using llc_size still shows better performance. tps for each test round(cluster size as factor in wake_wide): 1398.450887 1275.020401 1632.542437 1412.241627 1611.095692 1381.354294 1539.877146 avg tps = 1464 tps for each test round(llc size as factor in wake_wide): 1718.402983 1443.169823 1502.353823 1607.415861 1597.396924 1745.651814 1876.802168 avg tps = 1641 (+12%) so it seems using cluster_size as factor in "slave >= factor && master >= slave * factor" isn't a good choice for my machine at least. Thanks Barry _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel