linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Prateek Sood <prsood@codeaurora.org>
To: Peter Zijlstra <peterz@infradead.org>
Cc: tj@kernel.org, lizefan@huawei.com, cgroups@vger.kernel.org,
	mingo@kernel.org, longman@redhat.com,
	linux-kernel@vger.kernel.org, sramana@codeaurora.org,
	Thomas Gleixner <tglx@linutronix.de>
Subject: Re: [PATCH] cgroup/cpuset: remove circular dependency deadlock
Date: Thu, 7 Sep 2017 14:35:06 +0530	[thread overview]
Message-ID: <48cd3692-59fa-4e80-6aa5-25293f0fe113@codeaurora.org> (raw)
In-Reply-To: <20170907072848.2sjjddwincaeplju@hirez.programming.kicks-ass.net>

On 09/07/2017 12:58 PM, Peter Zijlstra wrote:
> On Thu, Sep 07, 2017 at 11:34:12AM +0530, Prateek Sood wrote:
>> Remove circular dependency deadlock in a scenario where hotplug of CPU is
>> being done while there is updation in cgroup and cpuset triggered from
>> userspace.
>>
>> Example scenario:
>> kworker/0:0 => kthreadd => init:729 => init:1 => kworker/0:0
>>
>> kworker/0:0 - percpu_down_write(&cpu_hotplug_lock)  [held]
>>               flush(work)   [no high prio workqueue available on CPU]
>>               wait_for_completion()
>>
>> kthreadd    - percpu_down_read(cgroup_threadgroup_rwsem)  [waiting]
>>
>> init:729    - percpu_down_write(cgroup_threadgroup_rwsem)   [held]
>>               lock(cpuset_mutex)   [waiting]
>>
>> init:1      - lock(cpuset_mutex)   [held]
>>               percpu_down_read(&cpu_hotplug_lock)   [waiting]
> 
> That's both unreadable and useless :/ You want to tell what code paths
> that were, not which random tasks happened to run them.
> 
> 
> 
>> Eliminate this dependecy by reordering locking of cpuset_mutex
>> and cpu_hotplug_lock in following order
>> 1. Acquire cpu_hotplug_lock (read)
>> 2. Acquire cpuset_mutex
>>
>> Signed-off-by: Prateek Sood <prsood@codeaurora.org>
>> ---
>>  kernel/cgroup/cpuset.c | 70 +++++++++++++++++++++++++++++++++++++++++++-------
>>  1 file changed, 61 insertions(+), 9 deletions(-)
>>
>> diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
>> index 2f4039b..687be57 100644
>> --- a/kernel/cgroup/cpuset.c
>> +++ b/kernel/cgroup/cpuset.c
>> @@ -843,10 +843,41 @@ static void rebuild_sched_domains_locked(void)
>>  out:
>>  	put_online_cpus();
>>  }
>> +
>> +/*
>> + * Rebuild scheduler domains.
>> + * Call with following lock held in the order
>> + * 1. cpu_hotplug_lock (read)
>> + * 2. cpuset_mutex
> 
> Do not put that in comments, nobody ever reads comments.
> 
>> + */
>> +static void rebuild_sched_domains_unlocked(void)
> 
> The common postfix for a function called with the cpuhotplug lock held
> is: _cpuslocked()
> 
>> +{
>> +        struct sched_domain_attr *attr;
>> +        cpumask_var_t *doms;
>> +        int ndoms;
> 
> 	lockdep_assert_cpus_held();
> 	lockdep_assert_held(&cpuset_mutex);
> 
>> +
>> +        /*
>> +         * We have raced with CPU hotplug. Don't do anything to avoid
>> +         * passing doms with offlined cpu to partition_sched_domains().
>> +         * Anyways, hotplug work item will rebuild sched domains.
>> +         */
>> +        if (!cpumask_equal(top_cpuset.effective_cpus, cpu_active_mask))
>> +                return;
>> +
>> +        /* Generate domain masks and attrs */
>> +        ndoms = generate_sched_domains(&doms, &attr);
>> +
>> +        /* Have scheduler rebuild the domains */
>> +        partition_sched_domains(ndoms, doms, attr);
>> +}
> 
> And you couldn't come up with a way to share _anything_ with the
> existing rebuild_sched_domains_locked() function?
> 
> *sigh*.. please try again.
> 

Thanks for the suggestion Peter, I will resend the patch

-- 
Qualcomm India Private Limited, on behalf of Qualcomm Innovation
Center, Inc., is a member of Code Aurora Forum, a Linux Foundation
Collaborative Project

  parent reply	other threads:[~2017-09-07  9:05 UTC|newest]

Thread overview: 19+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-09-07  6:04 [PATCH] cgroup/cpuset: remove circular dependency deadlock Prateek Sood
2017-09-07  7:28 ` Peter Zijlstra
2017-09-07  8:56   ` Boqun Feng
2017-09-07  9:07     ` Prateek Sood
2017-09-07  9:05   ` Prateek Sood [this message]
  -- strict thread matches above, loose matches on Subject: below --
2017-09-07 13:56 Prateek Sood
2017-09-07 17:45 ` Peter Zijlstra
2017-09-08  2:13   ` Prateek Sood
2017-09-07 17:51 ` Peter Zijlstra
2017-10-09 13:27   ` Prateek Sood
2017-10-11  9:48     ` Peter Zijlstra
2017-10-25  8:39       ` Prateek Sood
2017-10-25  9:30         ` Peter Zijlstra
2017-10-26 11:52           ` Prateek Sood
2017-10-26 14:05             ` Waiman Long
2017-10-27  8:03               ` Prateek Sood
2017-09-06 11:48 Prateek Sood
2017-09-06 12:56 ` Waiman Long
2017-09-06 14:23   ` Prateek Sood

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=48cd3692-59fa-4e80-6aa5-25293f0fe113@codeaurora.org \
    --to=prsood@codeaurora.org \
    --cc=cgroups@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=lizefan@huawei.com \
    --cc=longman@redhat.com \
    --cc=mingo@kernel.org \
    --cc=peterz@infradead.org \
    --cc=sramana@codeaurora.org \
    --cc=tglx@linutronix.de \
    --cc=tj@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).