Need help building for aarch64 kernel 5.2

Need help building for aarch64 kernel 5.2

[Dick Hollenbeck]

Hi Experts,

I really need some help please.

Here is the result of my first build attempt on a tree made from pristine
Linux 5.2.21

and the corresponding RT patch, which applied cleanly:



$ /i/runtime/arm64/kernel/build-arm64-rt.sh
make[1]: Entering directory '/ssd/build/linux-5.2.y'
  GEN     Makefile
scripts/kconfig/conf  --syncconfig Kconfig
  GEN     Makefile
  Using /ssd/linux-5.2.y as source for kernel
  CALL    /ssd/linux-5.2.y/scripts/atomic/check-atomics.sh
  CALL    /ssd/linux-5.2.y/scripts/checksyscalls.sh
  CHK     include/generated/compile.h
  GZIP    kernel/config_data.gz
  CC      kernel/configs.o
  AR      kernel/built-in.a
  GEN     .version
  CHK     include/generated/compile.h
  UPD     include/generated/compile.h
  CC      init/version.o
  AR      init/built-in.a
  LD      vmlinux.o
  MODPOST vmlinux.o
  MODINFO modules.builtin.modinfo
kernel/sched/core.o: In function `migrate_enable':
core.c:(.text+0x3750): undefined reference to `takedown_cpu_task'
core.c:(.text+0x3750): relocation truncated to fit: R_AARCH64_ADR_PREL_PG_HI21 against
undefined symbol `takedown_cpu_task'
core.c:(.text+0x3754): undefined reference to `takedown_cpu_task'
/ssd/linux-5.2.y/Makefile:1054: recipe for target 'vmlinux' failed
make[1]: *** [vmlinux] Error 1
make[1]: Leaving directory '/ssd/build/linux-5.2.y'
Makefile:179: recipe for target 'sub-make' failed
make: *** [sub-make] Error 2


Am I right to think that this pointer, takedown_cpu_task, is more that some maximum
distance away from object file "core.o" in the link image for the build setup?


How are other globals avoiding this same problem, what't the best fix?


Thanks much,

Dick

Re: Need help building for aarch64 kernel 5.2

[Paul Thomas]

On Tue, Nov 12, 2019 at 2:27 PM Dick Hollenbeck <dick@softplc.com> wrote:
>
> Hi Experts,
>
> I really need some help please.
>
> Here is the result of my first build attempt on a tree made from pristine
> Linux 5.2.21
>
> and the corresponding RT patch, which applied cleanly:
>
>
>
> $ /i/runtime/arm64/kernel/build-arm64-rt.sh
> make[1]: Entering directory '/ssd/build/linux-5.2.y'
>   GEN     Makefile
> scripts/kconfig/conf  --syncconfig Kconfig
>   GEN     Makefile
>   Using /ssd/linux-5.2.y as source for kernel
>   CALL    /ssd/linux-5.2.y/scripts/atomic/check-atomics.sh
>   CALL    /ssd/linux-5.2.y/scripts/checksyscalls.sh
>   CHK     include/generated/compile.h
>   GZIP    kernel/config_data.gz
>   CC      kernel/configs.o
>   AR      kernel/built-in.a
>   GEN     .version
>   CHK     include/generated/compile.h
>   UPD     include/generated/compile.h
>   CC      init/version.o
>   AR      init/built-in.a
>   LD      vmlinux.o
>   MODPOST vmlinux.o
>   MODINFO modules.builtin.modinfo
> kernel/sched/core.o: In function `migrate_enable':
> core.c:(.text+0x3750): undefined reference to `takedown_cpu_task'
> core.c:(.text+0x3750): relocation truncated to fit: R_AARCH64_ADR_PREL_PG_HI21 against
> undefined symbol `takedown_cpu_task'
> core.c:(.text+0x3754): undefined reference to `takedown_cpu_task'
> /ssd/linux-5.2.y/Makefile:1054: recipe for target 'vmlinux' failed
> make[1]: *** [vmlinux] Error 1
> make[1]: Leaving directory '/ssd/build/linux-5.2.y'
> Makefile:179: recipe for target 'sub-make' failed
> make: *** [sub-make] Error 2
>
>
> Am I right to think that this pointer, takedown_cpu_task, is more that some maximum
> distance away from object file "core.o" in the link image for the build setup?
>
>
> How are other globals avoiding this same problem, what't the best fix?


Hi Dick,

I've been building for arm64 for some time without issue. I generally
use git, but using the patch is fine too. Here are the general steps.

1. Download the base version. It's not on the kernel.org home page but
you can find it here (https://cdn.kernel.org/pub/linux/kernel/v5.x/)
 wget https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-5.2.21.tar.xz

2. Extract the kernel
 tar -xf linux-5.2.21.tar.xz

3. Change into the directory
 cd linux-5.2.21

4. Pull the patch. The main git devel page is a nice reference:
https://git.kernel.org/pub/scm/linux/kernel/git/rt/linux-rt-devel.git
If you look for the Commit messages with an [ANNOUNCE] prefix and
click on that such as "[ANNOUNCE] v5.2.21-rt13" then this brings up a
nice write-up Sebastian does with the URL of the RT patch. So in this
case the following command works:
wget https://cdn.kernel.org/pub/linux/kernel/projects/rt/5.2/older/patch-5.2.21-rt13.patch.xz

5. Extract the patch
 unxz patch-5.2.21-rt13.patch.xz

6. Apply the patch
 patch -p1 < patch-5.2.21-rt13.patch

7. Make the default config. This is where you customize the config to
turn on things like CONFIG_PREEMPT_RT_FULL. This is done with 'make
ARCH=arm64 menuconfig' or 'make ARCH=arm64 xconfig'
 make ARCH=arm64 defconfig

8. Check that you have a proper arm64 cross compiler in your PATH, the
exact prefix of your compiler may be different, but this is a good
test to see that we have the prefix correct for the next command.
 aarch64-linux-gnu-gcc --version

9. Build the Image (actual u-boot binary for arm64 kernels). The Image
file is located in arch/arm64/boot/
  make -j4 ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- Image

I hope this helps. Good luck!

-Paul

Re: Need help building for aarch64 kernel 5.2

[Dick Hollenbeck]

On 11/12/19 2:33 PM, Paul Thomas wrote:
> I hope this helps. Good luck!
> 
> -Paul


[PATCH] needed

Hi Paul,

A+ for your help.  Thanks very much for your valuable time and excellent post.

The problem is two fold:

1) the configuration variable CONFIG_HOTPLUG_CPU controls the existence of the
*definition* of 'takedown_cpu_task' at kernel/cpu.c line 880.  (The *declaration* is in a
header file.) But there's a usage at kernel/sched/core.c line 7399 and following line
which is not conditioned on that same CONFIG_HOTPLUG_CPU config variable.

3) The linker message is misleading.  All they had to say was that the symbol was
undefined.  The continued detail about this being an offset out of range issue created
ambiguity as to whether this was out of range first and foremost, leading to the undefined
status.  Clearly now it is simply an undefined usage.

I am looking for a low latency transfer of control from a GPIO interrupt handler that I
wrote in one of my drivers, all the way to a high RT priority wait_queue blocked user
thread.  And stock (non-RT) 5.1 kernel was atrocious, simply unbelievably bad.  Somewhere
along the way during the great hunt for improvement I turned off CONFIG_HOTPLUG_CPU just
to try a simpler config.


BTW cyclictest on that same stock 5.1 kernel was showing an intermittent 4 msecs of delay.
 Yes msecs, not usecs.


I would have submitted a patch for this but since its so trivial, it seems somebody with
commit rights could make this change faster than applying a patch.


Thanks all,

Dick

Re: Need help building for aarch64 kernel 5.2

[Paul Thomas]

On Wed, Nov 13, 2019 at 1:02 PM Dick Hollenbeck <dick@softplc.com> wrote:
>
> On 11/12/19 2:33 PM, Paul Thomas wrote:
> > I hope this helps. Good luck!
> >
> > -Paul
>
>
> [PATCH] needed
>
> Hi Paul,
>
> A+ for your help.  Thanks very much for your valuable time and excellent post.
>
> The problem is two fold:
>
> 1) the configuration variable CONFIG_HOTPLUG_CPU controls the existence of the
> *definition* of 'takedown_cpu_task' at kernel/cpu.c line 880.  (The *declaration* is in a
> header file.) But there's a usage at kernel/sched/core.c line 7399 and following line
> which is not conditioned on that same CONFIG_HOTPLUG_CPU config variable.
>
> 3) The linker message is misleading.  All they had to say was that the symbol was
> undefined.  The continued detail about this being an offset out of range issue created
> ambiguity as to whether this was out of range first and foremost, leading to the undefined
> status.  Clearly now it is simply an undefined usage.
>
> I am looking for a low latency transfer of control from a GPIO interrupt handler that I
> wrote in one of my drivers, all the way to a high RT priority wait_queue blocked user
> thread.  And stock (non-RT) 5.1 kernel was atrocious, simply unbelievably bad.  Somewhere
> along the way during the great hunt for improvement I turned off CONFIG_HOTPLUG_CPU just
> to try a simpler config.
>
>
> BTW cyclictest on that same stock 5.1 kernel was showing an intermittent 4 msecs of delay.
>  Yes msecs, not usecs.
>
>
> I would have submitted a patch for this but since its so trivial, it seems somebody with
> commit rights could make this change faster than applying a patch.

Yup, with CONFIG_HOTPLUG_CPU not set I get the same compile failure.
It looks like those two references to takedown_cpu_task should be
conditional on CONFIG_HOTPLUG_CPU like you said, but I'll leave that
to others to look at.

-Paul

[PATCH] sched/core: migrate_enable() must access takedown_cpu_task on !HOTPLUG_CPU

[Sebastian Andrzej Siewior]

The variable takedown_cpu_task is never declared/used on !HOTPLUG_CPU
except for migrate_enable(). This leads to a link error.

Don't use takedown_cpu_task in !HOTPLUG_CPU.

Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
---
 kernel/cpu.c        | 2 ++
 kernel/sched/core.c | 2 ++
 2 files changed, 4 insertions(+)

diff --git a/kernel/cpu.c b/kernel/cpu.c
index 123ad9626cc7f..81a6e5068972c 100644
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -887,7 +887,9 @@ static int take_cpu_down(void *_param)
 	return 0;
 }
 
+#ifdef CONFIG_PREEMPT_RT
 struct task_struct *takedown_cpu_task;
+#endif
 
 static int takedown_cpu(unsigned int cpu)
 {
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index bd26257eb2358..8bea013b2baf5 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -8212,9 +8212,11 @@ void migrate_enable(void)
 
 	p->migrate_disable = 0;
 	rq->nr_pinned--;
+#ifdef CONFIG_HOTPLUG_CPU
 	if (rq->nr_pinned == 0 && unlikely(!cpu_active(cpu)) &&
 	    takedown_cpu_task)
 		wake_up_process(takedown_cpu_task);
+#endif
 
 	if (!p->migrate_disable_scheduled)
 		goto out;
-- 
2.24.0