From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755239AbeEaNg1 (ORCPT ); Thu, 31 May 2018 09:36:27 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:45944 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1755053AbeEaNgZ (ORCPT ); Thu, 31 May 2018 09:36:25 -0400 Subject: Re: [PATCH v9 3/7] cpuset: Add cpuset.sched.load_balance flag to v2 To: Peter Zijlstra Cc: Tejun Heo , Li Zefan , Johannes Weiner , Ingo Molnar , cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, kernel-team@fb.com, pjt@google.com, luto@amacapital.net, Mike Galbraith , torvalds@linux-foundation.org, Roman Gushchin , Juri Lelli , Patrick Bellasi References: <1527601294-3444-1-git-send-email-longman@redhat.com> <1527601294-3444-4-git-send-email-longman@redhat.com> <20180531105416.GI12180@hirez.programming.kicks-ass.net> From: Waiman Long Organization: Red Hat Message-ID: Date: Thu, 31 May 2018 09:36:20 -0400 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.2.0 MIME-Version: 1.0 In-Reply-To: <20180531105416.GI12180@hirez.programming.kicks-ass.net> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Content-Language: en-US Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 05/31/2018 06:54 AM, Peter Zijlstra wrote: > On Tue, May 29, 2018 at 09:41:30AM -0400, Waiman Long wrote: > >> + cpuset.sched.load_balance >> + A read-write single value file which exists on non-root >> + cpuset-enabled cgroups. It is a binary value flag that accepts >> + either "0" (off) or "1" (on). This flag is set by the parent >> + and is not delegatable. It is on by default in the root cgroup. >> + >> + When it is on, tasks within this cpuset will be load-balanced >> + by the kernel scheduler. Tasks will be moved from CPUs with >> + high load to other CPUs within the same cpuset with less load >> + periodically. >> + >> + When it is off, there will be no load balancing among CPUs on >> + this cgroup. Tasks will stay in the CPUs they are running on >> + and will not be moved to other CPUs. > That is not entirely accurate I'm afraid (unless the patch makes it so, > I've yet to check). When you disable load-balancing on a cgroup you'll > get whatever balancing is left for the partition you happen to end up > in. > > Take for instance workqueue thingies, they use kthread_bind_mask() > (IIRC) and thus end up with PF_NO_SETAFFINITY so cpusets (or any other > cgroups really) do not have effect on them (long standing complaint). > > So take for instance the unbound numa enabled workqueue threads, those > will land in whatever partition and get balanced there. Thanks for the clarification. The patch doesn't make any changes in the scheduler. I was trying to say what the flag does. I will update the documentation about this nuisance. Cheers, Longman From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on archive.lwn.net X-Spam-Level: X-Spam-Status: No, score=-5.8 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham autolearn_force=no version=3.4.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by archive.lwn.net (Postfix) with ESMTP id 5A4D57D043 for ; Thu, 31 May 2018 13:36:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755117AbeEaNg0 (ORCPT ); Thu, 31 May 2018 09:36:26 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:45944 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1755053AbeEaNgZ (ORCPT ); Thu, 31 May 2018 09:36:25 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id D4852818BAEE; Thu, 31 May 2018 13:36:24 +0000 (UTC) Received: from llong.remote.csb (dhcp-17-81.bos.redhat.com [10.18.17.81]) by smtp.corp.redhat.com (Postfix) with ESMTP id 31E3F1DB25; Thu, 31 May 2018 13:36:21 +0000 (UTC) Subject: Re: [PATCH v9 3/7] cpuset: Add cpuset.sched.load_balance flag to v2 To: Peter Zijlstra Cc: Tejun Heo , Li Zefan , Johannes Weiner , Ingo Molnar , cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, kernel-team@fb.com, pjt@google.com, luto@amacapital.net, Mike Galbraith , torvalds@linux-foundation.org, Roman Gushchin , Juri Lelli , Patrick Bellasi References: <1527601294-3444-1-git-send-email-longman@redhat.com> <1527601294-3444-4-git-send-email-longman@redhat.com> <20180531105416.GI12180@hirez.programming.kicks-ass.net> From: Waiman Long Organization: Red Hat Message-ID: Date: Thu, 31 May 2018 09:36:20 -0400 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.2.0 MIME-Version: 1.0 In-Reply-To: <20180531105416.GI12180@hirez.programming.kicks-ass.net> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Content-Language: en-US X-Scanned-By: MIMEDefang 2.79 on 10.11.54.5 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.8]); Thu, 31 May 2018 13:36:24 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.8]); Thu, 31 May 2018 13:36:24 +0000 (UTC) for IP:'10.11.54.5' DOMAIN:'int-mx05.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'longman@redhat.com' RCPT:'' Sender: linux-doc-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-doc@vger.kernel.org On 05/31/2018 06:54 AM, Peter Zijlstra wrote: > On Tue, May 29, 2018 at 09:41:30AM -0400, Waiman Long wrote: > >> + cpuset.sched.load_balance >> + A read-write single value file which exists on non-root >> + cpuset-enabled cgroups. It is a binary value flag that accepts >> + either "0" (off) or "1" (on). This flag is set by the parent >> + and is not delegatable. It is on by default in the root cgroup. >> + >> + When it is on, tasks within this cpuset will be load-balanced >> + by the kernel scheduler. Tasks will be moved from CPUs with >> + high load to other CPUs within the same cpuset with less load >> + periodically. >> + >> + When it is off, there will be no load balancing among CPUs on >> + this cgroup. Tasks will stay in the CPUs they are running on >> + and will not be moved to other CPUs. > That is not entirely accurate I'm afraid (unless the patch makes it so, > I've yet to check). When you disable load-balancing on a cgroup you'll > get whatever balancing is left for the partition you happen to end up > in. > > Take for instance workqueue thingies, they use kthread_bind_mask() > (IIRC) and thus end up with PF_NO_SETAFFINITY so cpusets (or any other > cgroups really) do not have effect on them (long standing complaint). > > So take for instance the unbound numa enabled workqueue threads, those > will land in whatever partition and get balanced there. Thanks for the clarification. The patch doesn't make any changes in the scheduler. I was trying to say what the flag does. I will update the documentation about this nuisance. Cheers, Longman -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html