Re: poor cyclictest results with 5.0 series rt for arm64

From: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
To: Paul Thomas <pthomas8589@gmail.com>
Cc: Thomas Gleixner <tglx@linutronix.de>,
	linux-rt-users <linux-rt-users@vger.kernel.org>,
	dunc.mcphie@gmail.com, Anders Roxell <anders.roxell@linaro.org>
Subject: Re: poor cyclictest results with 5.0 series rt for arm64
Date: Wed, 14 Aug 2019 13:26:56 +0200	[thread overview]
Message-ID: <20190814112656.z3de2rgbqzpaquz5@linutronix.de> (raw)
In-Reply-To: <CAD56B7fqosJAhiCLikhcxv_3wY7ETOv++kmQcr-=9NKEGq7Agw@mail.gmail.com>

On 2019-08-13 14:02:39 [-0400], Paul Thomas wrote:
> I still get large latencies, like this:
> root@xu5:/opt/rt-tests# ./cyclictest -h ./cyclictest -S -m -n -p 99 -i
> 200 -h 400 -D 900
> # /dev/cpu_dma_latency set to 0us
> policy: fifo: loadavg: 0.10 0.10 0.03 1/134 1901
> 
> T: 0 ( 1886) P:99 I:200 C:2724618 Min:      5 Act:    6 Avg:    6 Max:      45
> T: 1 ( 1887) P:99 I:200 C:2724525 Min:      5 Act:    7 Avg:    6 Max:      29
> T: 2 ( 1888) P:99 I:200 C:2724246 Min:      5 Act:    6 Avg:    6 Max:    6338
> T: 3 ( 1889) P:99 I:200 C:2724339 Min:      5 Act:    8 Avg:    6 Max:     141
> 
> This is with the Lazy preempt commit in there.

I re-read the thread and figured out that you talk about 5.0 while I was
testing on 5.2. On my ARM64 box I end up with

|T: 0 ( 9577) P:99 I:250 C:19979015 Min:      3 Act:    4 Avg:    3 Max:      19                                                              
|T: 1 ( 9578) P:99 I:250 C:19978958 Min:      3 Act:    3 Avg:    3 Max:      24                                                              
|T: 2 ( 9579) P:99 I:250 C:19978901 Min:      3 Act:    4 Avg:    4 Max:      22                                                              
|T: 3 ( 9580) P:99 I:250 C:19978843 Min:      3 Act:    3 Avg:    3 Max:      19                                                              

with the patch I sent you. Without it, had spikes between 130us and
160us. Since you said it is not working, I looked more and came up with
this:

diff --git a/arch/arm64/include/asm/preempt.h b/arch/arm64/include/asm/preempt.h
index 3bfad251203b5..ca1c6fe8dd347 100644
--- a/arch/arm64/include/asm/preempt.h
+++ b/arch/arm64/include/asm/preempt.h
@@ -73,6 +73,8 @@ static inline bool __preempt_count_dec_and_test(void)
 	if (!pc || !READ_ONCE(ti->preempt_count))
 		return true;
 #ifdef CONFIG_PREEMPT_LAZY
+	if ((pc & ~PREEMPT_NEED_RESCHED))
+		return false;
 	if (current_thread_info()->preempt_lazy_count)
 		return false;
 	return test_thread_flag(TIF_NEED_RESCHED_LAZY);

but this shouldn't make any difference for you (but it is a bug fix).
Staring more into it, I don't see anything wrong.

v5.0 is not supported any more. Do you think, that you could re-test on
5.2-RT with those two patches I sent you?

> thanks,
> Paul

Sebastian
