From: Chengming Zhou <zhouchengming@bytedance.com>
To: Marek Szyprowski <m.szyprowski@samsung.com>,
mingo@redhat.com, peterz@infradead.org,
vincent.guittot@linaro.org, bristot@redhat.com,
zhaolei@cn.fujitsu.com, tj@kernel.org, lizefan.x@bytedance.com,
hannes@cmpxchg.org
Cc: linux-kernel@vger.kernel.org
Subject: Re: [External] Re: [PATCH v3 2/3] sched/cpuacct: optimize away RCU read lock
Date: Wed, 9 Mar 2022 11:08:39 +0800 [thread overview]
Message-ID: <7054def3-4f69-cf9f-546c-02a7435924d8@bytedance.com> (raw)
In-Reply-To: <f4bc652b-115f-35b5-91db-bad3b30fed9b@samsung.com>
On 2022/3/9 7:20 上午, Marek Szyprowski wrote:
> On 20.02.2022 06:14, Chengming Zhou wrote:
>> Since cpuacct_charge() is called from the scheduler update_curr(),
>> we must already have rq lock held, then the RCU read lock can
>> be optimized away.
>>
>> And do the same thing in it's wrapper cgroup_account_cputime(),
>> but we can't use lockdep_assert_rq_held() there, which defined
>> in kernel/sched/sched.h.
>>
>> Suggested-by: Peter Zijlstra (Intel) <peterz@infradead.org>
>> Signed-off-by: Chengming Zhou <zhouchengming@bytedance.com>
>
> This patch landed recently in linux-next as commit dc6e0818bc9a
> ("sched/cpuacct: Optimize away RCU read lock"). On my test systems I
> found that it triggers a following warning in the early boot stage:
Hi, thanks for the report. I've send a fix patch[1] for review.
[1] https://lore.kernel.org/lkml/20220305034103.57123-1-zhouchengming@bytedance.com/
>
> Calibrating delay loop (skipped), value calculated using timer
> frequency.. 48.00 BogoMIPS (lpj=240000)
> pid_max: default: 32768 minimum: 301
> Mount-cache hash table entries: 2048 (order: 1, 8192 bytes, linear)
> Mountpoint-cache hash table entries: 2048 (order: 1, 8192 bytes, linear)
> CPU: Testing write buffer coherency: ok
> CPU0: Spectre v2: using BPIALL workaround
>
> =============================
> WARNING: suspicious RCU usage
> 5.17.0-rc5-00050-gdc6e0818bc9a #11458 Not tainted
> -----------------------------
> ./include/linux/cgroup.h:481 suspicious rcu_dereference_check() usage!
>
> other info that might help us debug this:
>
>
> rcu_scheduler_active = 1, debug_locks = 1
> 2 locks held by kthreadd/2:
> #0: c1d7972c (&p->pi_lock){....}-{2:2}, at: task_rq_lock+0x30/0x118
> #1: ef7b52d0 (&rq->__lock){-...}-{2:2}, at:
> raw_spin_rq_lock_nested+0x24/0x34
>
> stack backtrace:
> CPU: 0 PID: 2 Comm: kthreadd Not tainted 5.17.0-rc5-00050-gdc6e0818bc9a
> #11458
> Hardware name: Samsung Exynos (Flattened Device Tree)
> unwind_backtrace from show_stack+0x10/0x14
> show_stack from dump_stack_lvl+0x58/0x70
> dump_stack_lvl from update_curr+0x1bc/0x35c
> update_curr from dequeue_task_fair+0xb0/0x8e8
> dequeue_task_fair from __do_set_cpus_allowed+0x19c/0x258
> __do_set_cpus_allowed from __set_cpus_allowed_ptr_locked+0x130/0x1d8
> __set_cpus_allowed_ptr_locked from __set_cpus_allowed_ptr+0x48/0x64
> __set_cpus_allowed_ptr from kthreadd+0x44/0x16c
> kthreadd from ret_from_fork+0x14/0x2c
> Exception stack(0xc1cb9fb0 to 0xc1cb9ff8)
> 9fa0: 00000000 00000000 00000000
> 00000000
> 9fc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000
> 00000000
> 9fe0: 00000000 00000000 00000000 00000000 00000013 00000000
> CPU0: thread -1, cpu 0, socket 9, mpidr 80000900
> cblist_init_generic: Setting adjustable number of callback queues.
> cblist_init_generic: Setting shift to 1 and lim to 1.
> Running RCU-tasks wait API self tests
> Setting up static identity map for 0x40100000 - 0x40100060
> rcu: Hierarchical SRCU implementation.
>
> =============================
> WARNING: suspicious RCU usage
> 5.17.0-rc5-00050-gdc6e0818bc9a #11458 Not tainted
> -----------------------------
> ./include/linux/cgroup.h:481 suspicious rcu_dereference_check() usage!
>
> other info that might help us debug this:
>
>
> rcu_scheduler_active = 1, debug_locks = 1
> 1 lock held by migration/0/13:
> #0: ef7b52d0 (&rq->__lock){-...}-{2:2}, at:
> raw_spin_rq_lock_nested+0x24/0x34
>
> stack backtrace:
> CPU: 0 PID: 13 Comm: migration/0 Not tainted
> 5.17.0-rc5-00050-gdc6e0818bc9a #11458
> Hardware name: Samsung Exynos (Flattened Device Tree)
> Stopper: 0x0 <- 0x0
> unwind_backtrace from show_stack+0x10/0x14
> show_stack from dump_stack_lvl+0x58/0x70
> dump_stack_lvl from put_prev_task_stop+0x16c/0x25c
> put_prev_task_stop from __schedule+0x698/0x964
> __schedule from schedule+0x54/0xe0
> schedule from smpboot_thread_fn+0x218/0x288
> smpboot_thread_fn from kthread+0xf0/0x134
> kthread from ret_from_fork+0x14/0x2c
> Exception stack(0xc1ccffb0 to 0xc1ccfff8)
> ffa0: 00000000 00000000 00000000
> 00000000
> ffc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000
> 00000000
> ffe0: 00000000 00000000 00000000 00000000 00000013 00000000
> smp: Bringing up secondary CPUs ...
> CPU1: thread -1, cpu 1, socket 9, mpidr 80000901
> CPU1: Spectre v2: using BPIALL workaround
> smp: Brought up 1 node, 2 CPUs
> SMP: Total of 2 processors activated (96.00 BogoMIPS).
>
> The above log comes from ARM 32bit Samsung Exnyos4210 based Trats board.
>
>> ---
>> include/linux/cgroup.h | 2 --
>> kernel/sched/cpuacct.c | 4 +---
>> 2 files changed, 1 insertion(+), 5 deletions(-)
>>
>> diff --git a/include/linux/cgroup.h b/include/linux/cgroup.h
>> index 75c151413fda..9a109c6ac0e0 100644
>> --- a/include/linux/cgroup.h
>> +++ b/include/linux/cgroup.h
>> @@ -791,11 +791,9 @@ static inline void cgroup_account_cputime(struct task_struct *task,
>>
>> cpuacct_charge(task, delta_exec);
>>
>> - rcu_read_lock();
>> cgrp = task_dfl_cgroup(task);
>> if (cgroup_parent(cgrp))
>> __cgroup_account_cputime(cgrp, delta_exec);
>> - rcu_read_unlock();
>> }
>>
>> static inline void cgroup_account_cputime_field(struct task_struct *task,
>> diff --git a/kernel/sched/cpuacct.c b/kernel/sched/cpuacct.c
>> index 307800586ac8..f79f88456d72 100644
>> --- a/kernel/sched/cpuacct.c
>> +++ b/kernel/sched/cpuacct.c
>> @@ -337,12 +337,10 @@ void cpuacct_charge(struct task_struct *tsk, u64 cputime)
>> unsigned int cpu = task_cpu(tsk);
>> struct cpuacct *ca;
>>
>> - rcu_read_lock();
>> + lockdep_assert_rq_held(cpu_rq(cpu));
>>
>> for (ca = task_ca(tsk); ca; ca = parent_ca(ca))
>> *per_cpu_ptr(ca->cpuusage, cpu) += cputime;
>> -
>> - rcu_read_unlock();
>> }
>>
>> /*
>
> Best regards
next prev parent reply other threads:[~2022-03-09 3:08 UTC|newest]
Thread overview: 18+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-02-20 5:14 [PATCH v3 1/3] sched/cpuacct: fix charge percpu cpuusage Chengming Zhou
2022-02-20 5:14 ` [PATCH v3 2/3] sched/cpuacct: optimize away RCU read lock Chengming Zhou
2022-03-01 15:24 ` [tip: sched/core] sched/cpuacct: Optimize " tip-bot2 for Chengming Zhou
[not found] ` <CGME20220308232034eucas1p2b0f39cee0f462af6004ebdfbe5bacb9f@eucas1p2.samsung.com>
2022-03-08 23:20 ` [PATCH v3 2/3] sched/cpuacct: optimize " Marek Szyprowski
2022-03-08 23:32 ` Peter Zijlstra
2022-03-08 23:44 ` Paul E. McKenney
2022-03-09 0:21 ` Paul E. McKenney
2022-03-10 8:45 ` Peter Zijlstra
2022-03-10 15:01 ` Paul E. McKenney
2022-03-12 12:15 ` Peter Zijlstra
2022-03-12 17:44 ` Paul E. McKenney
2022-03-09 3:08 ` Chengming Zhou [this message]
2022-02-20 5:14 ` [PATCH v3 3/3] sched/cpuacct: remove redundant " Chengming Zhou
2022-03-01 15:24 ` [tip: sched/core] sched/cpuacct: Remove " tip-bot2 for Chengming Zhou
[not found] ` <CGME20220308233107eucas1p119a2f5a8d4f5b5eec38ea8dde92b3368@eucas1p1.samsung.com>
2022-03-08 23:31 ` [PATCH v3 3/3] sched/cpuacct: remove " Marek Szyprowski
2022-02-22 18:01 ` [PATCH v3 1/3] sched/cpuacct: fix charge percpu cpuusage Tejun Heo
2022-02-23 9:11 ` Peter Zijlstra
2022-03-01 15:24 ` [tip: sched/core] sched/cpuacct: Fix " tip-bot2 for Chengming Zhou
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=7054def3-4f69-cf9f-546c-02a7435924d8@bytedance.com \
--to=zhouchengming@bytedance.com \
--cc=bristot@redhat.com \
--cc=hannes@cmpxchg.org \
--cc=linux-kernel@vger.kernel.org \
--cc=lizefan.x@bytedance.com \
--cc=m.szyprowski@samsung.com \
--cc=mingo@redhat.com \
--cc=peterz@infradead.org \
--cc=tj@kernel.org \
--cc=vincent.guittot@linaro.org \
--cc=zhaolei@cn.fujitsu.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).