* [PATCH] sched/rt:fix the missing of rt_rq runtime check in rt-period timer
@ 2020-12-05 3:38 carver4lio
2020-12-10 3:38 ` kernel test robot
0 siblings, 1 reply; 2+ messages in thread
From: carver4lio @ 2020-12-05 3:38 UTC (permalink / raw)
To: mingo
Cc: juri.lelli, vincent.guittot, dietmar.eggemann, rostedt, bsegall,
mgorman, bristot, linux-kernel, carver4lio, Hailong Liu
From: Hailong Liu <liu.hailong6@zte.com.cn>
The rq->rd->span of a cpu in a system with isolated cpus splited into two
different parts: one is for isolated cpus, another for non-isolated cpus.
When CONFIG_RT_GROUP_SCHED enabled, the handler of sched_rt_period_timer
updates rt_time and rt_runtime for every cpus in rq(this_cpu)->rd->span.
It means that other parts cpus out of this_cpu's rd->span will be missed
by sched_rt_period_timer handler, when CONFIG_RT_GROUP_SCHED enabled and
isolated cpus presents in system.
E.g problem will be triggered as follows on my 8 cores machine:
1 enable CONFIG_RT_GROUP_SCHED=y, and boot kernel with command-line
"isolcpus=4-7"
2 create a child group and init it:
mount -t cgroup -o cpu cpu /sys/fs/cgruop
mkdir /sys/fs/cgroup/child0
echo 950000 > /sys/fs/cgroup/child0/cpu.rt_runtime_us
3 run two rt-loop tasks, assume their pids are $pid1 and $pid2
4 affinity a rt task to the isolated cpu-sets
taskset -p 0xf0 $pid2
5 add tasks created above into child cpu-group
echo $pid1 > /sys/fs/cgroup/child0/tasks
echo $pid2 > /sys/fs/cgroup/child0/tasks
6 check wat happened:
"top": one of the task will fail to has cpu usage, but its stat is "R"
"kill": the task on the problem rt_rq can't be killed
This patch will fix this problem.
Signed-off-by: Hailong Liu <liu.hailong6@zte.com.cn>
---
kernel/sched/rt.c | 15 +++------------
1 file changed, 3 insertions(+), 12 deletions(-)
diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c
index 49ec096a8..c5c39695c 100644
--- a/kernel/sched/rt.c
+++ b/kernel/sched/rt.c
@@ -855,19 +855,10 @@ static int do_sched_rt_period_timer(struct rt_bandwidth *rt_b, int overrun)
int i, idle = 1, throttled = 0;
const struct cpumask *span;
- span = sched_rt_period_mask();
#ifdef CONFIG_RT_GROUP_SCHED
- /*
- * FIXME: isolated CPUs should really leave the root task group,
- * whether they are isolcpus or were isolated via cpusets, lest
- * the timer run on a CPU which does not service all runqueues,
- * potentially leaving other CPUs indefinitely throttled. If
- * isolation is really required, the user will turn the throttle
- * off to kill the perturbations it causes anyway. Meanwhile,
- * this maintains functionality for boot and/or troubleshooting.
- */
- if (rt_b == &root_task_group.rt_bandwidth)
- span = cpu_online_mask;
+ span = cpu_online_mask;
+#else
+ span = sched_rt_period_mask();
#endif
for_each_cpu(i, span) {
int enqueue = 0;
--
2.17.1
^ permalink raw reply related [flat|nested] 2+ messages in thread
* Re: [PATCH] sched/rt:fix the missing of rt_rq runtime check in rt-period timer
2020-12-05 3:38 [PATCH] sched/rt:fix the missing of rt_rq runtime check in rt-period timer carver4lio
@ 2020-12-10 3:38 ` kernel test robot
0 siblings, 0 replies; 2+ messages in thread
From: kernel test robot @ 2020-12-10 3:38 UTC (permalink / raw)
To: carver4lio, mingo
Cc: kbuild-all, clang-built-linux, juri.lelli, vincent.guittot,
dietmar.eggemann, rostedt, bsegall, mgorman, bristot,
linux-kernel, carver4lio
[-- Attachment #1: Type: text/plain, Size: 11474 bytes --]
Hi,
Thank you for the patch! Perhaps something to improve:
[auto build test WARNING on tip/sched/core]
[also build test WARNING on v5.10-rc7 next-20201209]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]
url: https://github.com/0day-ci/linux/commits/carver4lio-163-com/sched-rt-fix-the-missing-of-rt_rq-runtime-check-in-rt-period-timer/20201205-142057
base: https://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git 21bf7cbd1b100758cc82f5340576028d3d83119b
config: mips-randconfig-r026-20201209 (attached as .config)
compiler: clang version 12.0.0 (https://github.com/llvm/llvm-project 1968804ac726e7674d5de22bc2204b45857da344)
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# install mips cross compiling tool for clang build
# apt-get install binutils-mips-linux-gnu
# https://github.com/0day-ci/linux/commit/65b78641992cb9b9ae024d4b9ee02822a7d45167
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review carver4lio-163-com/sched-rt-fix-the-missing-of-rt_rq-runtime-check-in-rt-period-timer/20201205-142057
git checkout 65b78641992cb9b9ae024d4b9ee02822a7d45167
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross ARCH=mips
If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>
All warnings (new ones prefixed by >>):
kernel/sched/rt.c:669:6: warning: no previous prototype for function 'sched_rt_bandwidth_account'
bool sched_rt_bandwidth_account(struct rt_rq
^
kernel/sched/rt.c:669:1: note: declare 'static' if the function is not intended to be used outside of this translation unit
bool sched_rt_bandwidth_account(struct rt_rq
^
static
>> kernel/sched/rt.c:583:37: warning: unused function 'sched_rt_period_mask'
static inline const struct cpumask
^
fatal error: error in backend: Nested variants found in inline asm string: ' .set push
.set mips64r2
.if ( 0x00 ) != -1)) 0x00 ) != -1)) : ($( static struct ftrace_branch_data __attribute__((__aligned__(4))) __attribute__((__section__("_ftrace_branch"))) __if_trace = $( .func = __func__, .file = "arch/mips/include/asm/bitops.h", .line = 105, $); 0x00 ) != -1)) : $))) ) && ( 0 ); .set push; .set mips64r2; .rept 1; sync 0x00; .endr; .set pop; .else; ; .endif
1: ll $0, $1
or $0, $2
sc $0, $1
beqz $0, 1b
.set pop
'
clang-12: error: clang frontend command failed with exit code 70 (use -v to see invocation)
clang version 12.0.0 (git://gitmirror/llvm_project 1968804ac726e7674d5de22bc2204b45857da344)
Target: mipsel-unknown-linux-gnu
Thread model: posix
InstalledDir: /opt/cross/clang-1968804ac7/bin
clang-12: note: diagnostic msg:
Makefile arch drivers include kernel scripts source usr
vim +/sched_rt_period_mask +583 kernel/sched/rt.c
23b0fdfc9299b1 kernel/sched_rt.c Peter Zijlstra 2008-02-13 581
d0b27fa77854b1 kernel/sched_rt.c Peter Zijlstra 2008-04-19 582 #ifdef CONFIG_SMP
c6c4927b22a351 kernel/sched_rt.c Rusty Russell 2008-11-25 @583 static inline const struct cpumask *sched_rt_period_mask(void)
d0b27fa77854b1 kernel/sched_rt.c Peter Zijlstra 2008-04-19 584 {
424c93fe4cbe71 kernel/sched/rt.c Nathan Zimmer 2013-05-09 585 return this_rq()->rd->span;
d0b27fa77854b1 kernel/sched_rt.c Peter Zijlstra 2008-04-19 586 }
6f505b16425a51 kernel/sched_rt.c Peter Zijlstra 2008-01-25 587 #else
c6c4927b22a351 kernel/sched_rt.c Rusty Russell 2008-11-25 588 static inline const struct cpumask *sched_rt_period_mask(void)
d0b27fa77854b1 kernel/sched_rt.c Peter Zijlstra 2008-04-19 589 {
c6c4927b22a351 kernel/sched_rt.c Rusty Russell 2008-11-25 590 return cpu_online_mask;
d0b27fa77854b1 kernel/sched_rt.c Peter Zijlstra 2008-04-19 591 }
d0b27fa77854b1 kernel/sched_rt.c Peter Zijlstra 2008-04-19 592 #endif
6f505b16425a51 kernel/sched_rt.c Peter Zijlstra 2008-01-25 593
d0b27fa77854b1 kernel/sched_rt.c Peter Zijlstra 2008-04-19 594 static inline
d0b27fa77854b1 kernel/sched_rt.c Peter Zijlstra 2008-04-19 595 struct rt_rq *sched_rt_period_rt_rq(struct rt_bandwidth *rt_b, int cpu)
6f505b16425a51 kernel/sched_rt.c Peter Zijlstra 2008-01-25 596 {
d0b27fa77854b1 kernel/sched_rt.c Peter Zijlstra 2008-04-19 597 return container_of(rt_b, struct task_group, rt_bandwidth)->rt_rq[cpu];
d0b27fa77854b1 kernel/sched_rt.c Peter Zijlstra 2008-04-19 598 }
d0b27fa77854b1 kernel/sched_rt.c Peter Zijlstra 2008-04-19 599
ac086bc22997a2 kernel/sched_rt.c Peter Zijlstra 2008-04-19 600 static inline struct rt_bandwidth *sched_rt_bandwidth(struct rt_rq *rt_rq)
ac086bc22997a2 kernel/sched_rt.c Peter Zijlstra 2008-04-19 601 {
ac086bc22997a2 kernel/sched_rt.c Peter Zijlstra 2008-04-19 602 return &rt_rq->tg->rt_bandwidth;
ac086bc22997a2 kernel/sched_rt.c Peter Zijlstra 2008-04-19 603 }
ac086bc22997a2 kernel/sched_rt.c Peter Zijlstra 2008-04-19 604
55e12e5e7b1d7e kernel/sched_rt.c Dhaval Giani 2008-06-24 605 #else /* !CONFIG_RT_GROUP_SCHED */
9f0c1e560c4332 kernel/sched_rt.c Peter Zijlstra 2008-02-13 606
d0b27fa77854b1 kernel/sched_rt.c Peter Zijlstra 2008-04-19 607 static inline u64 sched_rt_runtime(struct rt_rq *rt_rq)
d0b27fa77854b1 kernel/sched_rt.c Peter Zijlstra 2008-04-19 608 {
ac086bc22997a2 kernel/sched_rt.c Peter Zijlstra 2008-04-19 609 return rt_rq->rt_runtime;
ac086bc22997a2 kernel/sched_rt.c Peter Zijlstra 2008-04-19 610 }
ac086bc22997a2 kernel/sched_rt.c Peter Zijlstra 2008-04-19 611
ac086bc22997a2 kernel/sched_rt.c Peter Zijlstra 2008-04-19 612 static inline u64 sched_rt_period(struct rt_rq *rt_rq)
ac086bc22997a2 kernel/sched_rt.c Peter Zijlstra 2008-04-19 613 {
ac086bc22997a2 kernel/sched_rt.c Peter Zijlstra 2008-04-19 614 return ktime_to_ns(def_rt_bandwidth.rt_period);
6f505b16425a51 kernel/sched_rt.c Peter Zijlstra 2008-01-25 615 }
6f505b16425a51 kernel/sched_rt.c Peter Zijlstra 2008-01-25 616
ec514c487c3d4b kernel/sched_rt.c Cheng Xu 2011-05-14 617 typedef struct rt_rq *rt_rq_iter_t;
ec514c487c3d4b kernel/sched_rt.c Cheng Xu 2011-05-14 618
ec514c487c3d4b kernel/sched_rt.c Cheng Xu 2011-05-14 619 #define for_each_rt_rq(rt_rq, iter, rq) \
ec514c487c3d4b kernel/sched_rt.c Cheng Xu 2011-05-14 620 for ((void) iter, rt_rq = &rq->rt; rt_rq; rt_rq = NULL)
ec514c487c3d4b kernel/sched_rt.c Cheng Xu 2011-05-14 621
6f505b16425a51 kernel/sched_rt.c Peter Zijlstra 2008-01-25 622 #define for_each_sched_rt_entity(rt_se) \
6f505b16425a51 kernel/sched_rt.c Peter Zijlstra 2008-01-25 623 for (; rt_se; rt_se = NULL)
6f505b16425a51 kernel/sched_rt.c Peter Zijlstra 2008-01-25 624
6f505b16425a51 kernel/sched_rt.c Peter Zijlstra 2008-01-25 625 static inline struct rt_rq *group_rt_rq(struct sched_rt_entity *rt_se)
6f505b16425a51 kernel/sched_rt.c Peter Zijlstra 2008-01-25 626 {
6f505b16425a51 kernel/sched_rt.c Peter Zijlstra 2008-01-25 627 return NULL;
6f505b16425a51 kernel/sched_rt.c Peter Zijlstra 2008-01-25 628 }
6f505b16425a51 kernel/sched_rt.c Peter Zijlstra 2008-01-25 629
9f0c1e560c4332 kernel/sched_rt.c Peter Zijlstra 2008-02-13 630 static inline void sched_rt_rq_enqueue(struct rt_rq *rt_rq)
6f505b16425a51 kernel/sched_rt.c Peter Zijlstra 2008-01-25 631 {
f4ebcbc0d7e009 kernel/sched/rt.c Kirill Tkhai 2014-03-15 632 struct rq *rq = rq_of_rt_rq(rt_rq);
f4ebcbc0d7e009 kernel/sched/rt.c Kirill Tkhai 2014-03-15 633
f4ebcbc0d7e009 kernel/sched/rt.c Kirill Tkhai 2014-03-15 634 if (!rt_rq->rt_nr_running)
f4ebcbc0d7e009 kernel/sched/rt.c Kirill Tkhai 2014-03-15 635 return;
f4ebcbc0d7e009 kernel/sched/rt.c Kirill Tkhai 2014-03-15 636
f4ebcbc0d7e009 kernel/sched/rt.c Kirill Tkhai 2014-03-15 637 enqueue_top_rt_rq(rt_rq);
8875125efe8402 kernel/sched/rt.c Kirill Tkhai 2014-06-29 638 resched_curr(rq);
6f505b16425a51 kernel/sched_rt.c Peter Zijlstra 2008-01-25 639 }
6f505b16425a51 kernel/sched_rt.c Peter Zijlstra 2008-01-25 640
9f0c1e560c4332 kernel/sched_rt.c Peter Zijlstra 2008-02-13 641 static inline void sched_rt_rq_dequeue(struct rt_rq *rt_rq)
6f505b16425a51 kernel/sched_rt.c Peter Zijlstra 2008-01-25 642 {
f4ebcbc0d7e009 kernel/sched/rt.c Kirill Tkhai 2014-03-15 643 dequeue_top_rt_rq(rt_rq);
6f505b16425a51 kernel/sched_rt.c Peter Zijlstra 2008-01-25 644 }
6f505b16425a51 kernel/sched_rt.c Peter Zijlstra 2008-01-25 645
46383648b3c769 kernel/sched/rt.c Kirill Tkhai 2014-03-15 646 static inline int rt_rq_throttled(struct rt_rq *rt_rq)
46383648b3c769 kernel/sched/rt.c Kirill Tkhai 2014-03-15 647 {
46383648b3c769 kernel/sched/rt.c Kirill Tkhai 2014-03-15 648 return rt_rq->rt_throttled;
46383648b3c769 kernel/sched/rt.c Kirill Tkhai 2014-03-15 649 }
46383648b3c769 kernel/sched/rt.c Kirill Tkhai 2014-03-15 650
c6c4927b22a351 kernel/sched_rt.c Rusty Russell 2008-11-25 651 static inline const struct cpumask *sched_rt_period_mask(void)
d0b27fa77854b1 kernel/sched_rt.c Peter Zijlstra 2008-04-19 652 {
c6c4927b22a351 kernel/sched_rt.c Rusty Russell 2008-11-25 653 return cpu_online_mask;
d0b27fa77854b1 kernel/sched_rt.c Peter Zijlstra 2008-04-19 654 }
d0b27fa77854b1 kernel/sched_rt.c Peter Zijlstra 2008-04-19 655
d0b27fa77854b1 kernel/sched_rt.c Peter Zijlstra 2008-04-19 656 static inline
d0b27fa77854b1 kernel/sched_rt.c Peter Zijlstra 2008-04-19 657 struct rt_rq *sched_rt_period_rt_rq(struct rt_bandwidth *rt_b, int cpu)
d0b27fa77854b1 kernel/sched_rt.c Peter Zijlstra 2008-04-19 658 {
d0b27fa77854b1 kernel/sched_rt.c Peter Zijlstra 2008-04-19 659 return &cpu_rq(cpu)->rt;
d0b27fa77854b1 kernel/sched_rt.c Peter Zijlstra 2008-04-19 660 }
d0b27fa77854b1 kernel/sched_rt.c Peter Zijlstra 2008-04-19 661
ac086bc22997a2 kernel/sched_rt.c Peter Zijlstra 2008-04-19 662 static inline struct rt_bandwidth *sched_rt_bandwidth(struct rt_rq *rt_rq)
ac086bc22997a2 kernel/sched_rt.c Peter Zijlstra 2008-04-19 663 {
ac086bc22997a2 kernel/sched_rt.c Peter Zijlstra 2008-04-19 664 return &def_rt_bandwidth;
ac086bc22997a2 kernel/sched_rt.c Peter Zijlstra 2008-04-19 665 }
ac086bc22997a2 kernel/sched_rt.c Peter Zijlstra 2008-04-19 666
55e12e5e7b1d7e kernel/sched_rt.c Dhaval Giani 2008-06-24 667 #endif /* CONFIG_RT_GROUP_SCHED */
d0b27fa77854b1 kernel/sched_rt.c Peter Zijlstra 2008-04-19 668
faa5993736d9b4 kernel/sched/rt.c Juri Lelli 2014-02-21 @669 bool sched_rt_bandwidth_account(struct rt_rq *rt_rq)
faa5993736d9b4 kernel/sched/rt.c Juri Lelli 2014-02-21 670 {
faa5993736d9b4 kernel/sched/rt.c Juri Lelli 2014-02-21 671 struct rt_bandwidth *rt_b = sched_rt_bandwidth(rt_rq);
faa5993736d9b4 kernel/sched/rt.c Juri Lelli 2014-02-21 672
faa5993736d9b4 kernel/sched/rt.c Juri Lelli 2014-02-21 673 return (hrtimer_active(&rt_b->rt_period_timer) ||
faa5993736d9b4 kernel/sched/rt.c Juri Lelli 2014-02-21 674 rt_rq->rt_time < rt_b->rt_runtime);
faa5993736d9b4 kernel/sched/rt.c Juri Lelli 2014-02-21 675 }
faa5993736d9b4 kernel/sched/rt.c Juri Lelli 2014-02-21 676
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org
[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 26899 bytes --]
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2020-12-10 3:40 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-12-05 3:38 [PATCH] sched/rt:fix the missing of rt_rq runtime check in rt-period timer carver4lio
2020-12-10 3:38 ` kernel test robot
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).