From: Suren Baghdasaryan <surenb@google.com>
To: Peter Zijlstra <peterz@infradead.org>
Cc: Ingo Molnar <mingo@redhat.com>,
Johannes Weiner <hannes@cmpxchg.org>,
will@kernel.org, Andrew Morton <akpm@linux-foundation.org>,
Thomas Gleixner <tglx@linutronix.de>,
ben.dooks@codethink.co.uk, cl@rock-chips.com, ke.wang@unisoc.com,
Shakeel Butt <shakeelb@google.com>,
LKML <linux-kernel@vger.kernel.org>,
kernel-team <kernel-team@android.com>
Subject: Re: [PATCH 1/1] kthread: break dependency between worker->lock and task_struct->pi_lock
Date: Tue, 28 Apr 2020 11:04:14 -0700 [thread overview]
Message-ID: <CAJuCfpGsRwJGwyny0X0APa2yCjyBttyMtTOPr3K5NqcWehtzxw@mail.gmail.com> (raw)
In-Reply-To: <20200428163125.GC16910@hirez.programming.kicks-ass.net>
On Tue, Apr 28, 2020 at 9:31 AM Peter Zijlstra <peterz@infradead.org> wrote:
>
> On Mon, Apr 27, 2020 at 11:43:58AM -0700, Suren Baghdasaryan wrote:
> > A number of kthread-related functions indirectly take task_struct->pi_lock
> > while holding worker->lock in the call chain like this:
> > spin_lock(&worker->lock)
> > kthread_insert_work
> > wake_up_process
> > try_to_wake_up
> > raw_spin_lock_irqsave(&p->pi_lock, flags)
> >
> > This lock dependency exists whenever kthread_insert_work is called either
> > directly or indirectly via __kthread_queue_delayed_work in the following
> > functions:
> > kthread_queue_work
> > kthread_delayed_work_timer_fn
> > kthread_queue_delayed_work
> > kthread_flush_work
> > kthread_mod_delayed_work
> >
> > This creates possibilities for circular dependencies like the one reported
> > at: https://lkml.org/lkml/2020/4/24/954
>
> Please, do not use lkml.org links.
Thanks for the review! Would
https://lore.kernel.org/lkml/CAJuCfpG4NkhpQvZjgXZ_3gm6Hf1QgN_eUOQ8iX9Cv1k9whLwSQ@mail.gmail.com
be better or should I just add the body of that report here? Or do not
mention it at all?
>
> Also, ideally, we'd pull that kthread_queue_delayed_work() out from
> under rq->lock.
I understand but I don't see an easy way to do that. We need to start
PSI polling whenever a monitored PSI state changes:
https://elixir.bootlin.com/linux/v5.6.7/source/kernel/sched/psi.c#L783.
This is happening under rq->lock because PSI accounting is done from
inside enqueue_task/dequeue_task - the call chain is:
enqueue_task > psi_enqueue > psi_task_change > psi_group_change >
psi_schedule_poll_work > psi_task_change
IIUC enqueue_task/dequeue_task are called with rq->lock taken, so
moving kthread_queue_delayed_work out is not trivial.
>
> In fact, looking at it, WTH is the delayed branch of
> kthread_queue_delayed_work() under that lock? That whole
> delayed_work_list thing smells like bong-hits.
I have the poll_scheduled atomic specifically to ensure that
kthread_queue_delayed_work does not block as commented here:
https://elixir.bootlin.com/linux/v5.7-rc3/source/kernel/sched/psi.c#L551.
I understand this is not ideal. If there is a better way to schedule
that kworker while ensuring it does not block I would be happy to
rework this. Any suggestions?
>
> --
> To unsubscribe from this group and stop receiving emails from it, send an email to kernel-team+unsubscribe@android.com.
>
next prev parent reply other threads:[~2020-04-28 18:04 UTC|newest]
Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-04-27 18:43 [PATCH 1/1] kthread: break dependency between worker->lock and task_struct->pi_lock Suren Baghdasaryan
2020-04-28 16:31 ` Peter Zijlstra
2020-04-28 18:04 ` Suren Baghdasaryan [this message]
2020-04-30 17:57 ` Suren Baghdasaryan
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CAJuCfpGsRwJGwyny0X0APa2yCjyBttyMtTOPr3K5NqcWehtzxw@mail.gmail.com \
--to=surenb@google.com \
--cc=akpm@linux-foundation.org \
--cc=ben.dooks@codethink.co.uk \
--cc=cl@rock-chips.com \
--cc=hannes@cmpxchg.org \
--cc=ke.wang@unisoc.com \
--cc=kernel-team@android.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mingo@redhat.com \
--cc=peterz@infradead.org \
--cc=shakeelb@google.com \
--cc=tglx@linutronix.de \
--cc=will@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).