* [Buildroot] [RFC next 1/2] linux: fix builds for kernels < 5.6 and host-gcc >= 10
@ 2022-08-22 22:14 Julien Olivain
2022-08-22 22:14 ` [Buildroot] [RFC next 2/2] support/testing/tests/core/test_kernel3_10: new test Julien Olivain
2022-12-31 19:33 ` [Buildroot] [RFC next 1/2] linux: fix builds for kernels < 5.6 and host-gcc >= 10 Yann E. MORIN
0 siblings, 2 replies; 5+ messages in thread
From: Julien Olivain @ 2022-08-22 22:14 UTC (permalink / raw)
To: buildroot; +Cc: Julien Olivain
During a linux-backports update, it was found that kernel v3.10.108 was
failing to compile. See:
https://lists.buildroot.org/pipermail/buildroot/2022-August/649507.html
This issue was introduced by commit 9b41b54be07711c10ad13ce157be272ed1cf402e
"linux: fix build with host-gcc 10+", which removes all declarations of
"yylloc" symbols in the dtc parser. This symbol is generated by bison, if the
"%locations" directive is provided in the parser. See:
https://git.savannah.gnu.org/cgit/bison.git/tree/doc/bison.texi?h=v3.8.2#n5984
Kernel versions < 5.6 did not include this directive, so removing all
yylloc declararions in the parser also was failing for those version.
In the kernel, dtc was updated to v1.5.1-22-gc40aeb60b47a in:
https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=0cec114e36606412908a35695a5db944cec2e3db
This commit is included in kernel v5.6.
This dtc update include the dtc commit:
https://git.kernel.org/pub/scm/utils/dtc/dtc.git/commit/?id=7150286225476345bd6e7312331e3baf4d621c32
which adds the '%locations' directive.
This commit fixes the issue by programmatically adding the '%locations'
Bison directive, if it's not found in the parser file.
Fixes:
- https://bugs.busybox.net/show_bug.cgi?id=14971
Signed-off-by: Julien Olivain <ju.o@free.fr>
---
Tested by:
- Building a recent kernel:
qemu_aarch64_virt_defconfig (Kernel v5.15.18)
- Building an old kernel:
qemu_arm_vexpress_defconfig with kernel 3.10.108
---
linux/linux.mk | 14 ++++++++++++++
1 file changed, 14 insertions(+)
diff --git a/linux/linux.mk b/linux/linux.mk
index 3d9ac37959..f3eb9c32cb 100644
--- a/linux/linux.mk
+++ b/linux/linux.mk
@@ -155,6 +155,7 @@ LINUX_MAKE_FLAGS = \
INSTALL_MOD_PATH=$(TARGET_DIR) \
CROSS_COMPILE="$(TARGET_CROSS)" \
WERROR=0 \
+ REGENERATE_PARSERS=1 \
DEPMOD=$(HOST_DIR)/sbin/depmod
ifeq ($(BR2_REPRODUCIBLE),y)
@@ -286,6 +287,19 @@ define LINUX_DROP_YYLLOC
endef
LINUX_POST_PATCH_HOOKS += LINUX_DROP_YYLLOC
+# Kernel version < 5.6 breaks if host-gcc version is >= 10 and
+# 'yylloc' symbol is removed in previous hook, due to missing
+# '%locations' bison directive in dtc-parser.y. See:
+# https://bugs.busybox.net/show_bug.cgi?id=14971
+define LINUX_ADD_DTC_LOCATIONS
+ $(Q)DTC_PARSER=$(@D)/scripts/dtc/dtc-parser.y; \
+ if test -e "$${DTC_PARSER}" \
+ && ! grep -Eq '^%locations$$' "$${DTC_PARSER}" ; then \
+ $(SED) '/^%{$$/i %locations' "$${DTC_PARSER}"; \
+ fi
+endef
+LINUX_POST_PATCH_HOOKS += LINUX_ADD_DTC_LOCATIONS
+
# Older linux kernels use deprecated perl constructs in timeconst.pl
# that were removed for perl 5.22+ so it breaks on newer distributions
# Try a dry-run patch to see if this applies, if it does go ahead
--
2.37.2
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot
^ permalink raw reply related [flat|nested] 5+ messages in thread
* [Buildroot] [RFC next 2/2] support/testing/tests/core/test_kernel3_10: new test
2022-08-22 22:14 [Buildroot] [RFC next 1/2] linux: fix builds for kernels < 5.6 and host-gcc >= 10 Julien Olivain
@ 2022-08-22 22:14 ` Julien Olivain
2022-12-31 19:38 ` Yann E. MORIN
2022-12-31 19:33 ` [Buildroot] [RFC next 1/2] linux: fix builds for kernels < 5.6 and host-gcc >= 10 Yann E. MORIN
1 sibling, 1 reply; 5+ messages in thread
From: Julien Olivain @ 2022-08-22 22:14 UTC (permalink / raw)
To: buildroot; +Cc: Julien Olivain
This commit add a test to check that a relatively old 3.10 kernel are
building successfully in Buildroot. This test was written for
investigating the bug described in:
- https://bugs.busybox.net/show_bug.cgi?id=14971
Signed-off-by: Julien Olivain <ju.o@free.fr>
---
Tested with:
python3 -m flake8 \
support/testing/tests/core/test_kernel3_10.py
[no-output]
support/testing/run-tests \
-d dl \
-o output_folder tests.core.test_kernel3_10.TestOldKernel3_10
...
OK
---
DEVELOPERS | 1 +
support/testing/tests/core/test_kernel3_10.py | 43 +++++++++++++++++++
2 files changed, 44 insertions(+)
create mode 100644 support/testing/tests/core/test_kernel3_10.py
diff --git a/DEVELOPERS b/DEVELOPERS
index 4287fd9cf7..5cf63e919e 100644
--- a/DEVELOPERS
+++ b/DEVELOPERS
@@ -1680,6 +1680,7 @@ F: package/python-gnupg/
F: package/python-pyalsa/
F: package/riscv-isa-sim/
F: package/zynaddsubfx/
+F: support/testing/tests/core/test_kernel3_10.py
F: support/testing/tests/package/sample_python_distro.py
F: support/testing/tests/package/sample_python_gnupg.py
F: support/testing/tests/package/sample_python_pyalsa.py
diff --git a/support/testing/tests/core/test_kernel3_10.py b/support/testing/tests/core/test_kernel3_10.py
new file mode 100644
index 0000000000..5ff5ed1cad
--- /dev/null
+++ b/support/testing/tests/core/test_kernel3_10.py
@@ -0,0 +1,43 @@
+import os
+
+import infra.basetest
+
+
+# For testing the compilation of an old 3.10 kernel, we use a armv7
+# gcc5 bootlin toolchain, which also includes 3.10 kernel headers.
+class TestOldKernel3_10(infra.basetest.BRTest):
+ config = \
+ """
+ BR2_arm=y
+ BR2_cortex_a9=y
+ BR2_ARM_EABIHF=y
+ BR2_ARM_ENABLE_NEON=y
+ BR2_ARM_ENABLE_VFP=y
+ BR2_LINUX_KERNEL=y
+ BR2_LINUX_KERNEL_CUSTOM_VERSION=y
+ BR2_LINUX_KERNEL_CUSTOM_VERSION_VALUE="3.10.108"
+ BR2_LINUX_KERNEL_DEFCONFIG="vexpress"
+ BR2_LINUX_KERNEL_DTS_SUPPORT=y
+ BR2_LINUX_KERNEL_INTREE_DTS_NAME="vexpress-v2p-ca9"
+ BR2_TARGET_GENERIC_GETTY_PORT="ttyAMA0"
+ BR2_TARGET_ROOTFS_CPIO=y
+ BR2_TARGET_ROOTFS_CPIO_GZIP=y
+ BR2_TOOLCHAIN_EXTERNAL=y
+ BR2_TOOLCHAIN_EXTERNAL_CUSTOM=y
+ BR2_TOOLCHAIN_EXTERNAL_CUSTOM_GLIBC=y
+ BR2_TOOLCHAIN_EXTERNAL_CXX=y
+ BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y
+ BR2_TOOLCHAIN_EXTERNAL_GCC_5=y
+ BR2_TOOLCHAIN_EXTERNAL_HEADERS_3_10=y
+ BR2_TOOLCHAIN_EXTERNAL_URL="https://toolchains.bootlin.com/downloads/releases/toolchains/armv7-eabihf/tarballs/armv7-eabihf--glibc--stable-2017.05-toolchains-1-1.tar.bz2"
+ """
+
+ def test_run(self):
+ initrd = os.path.join(self.builddir, "images", "rootfs.cpio.gz")
+ kern = os.path.join(self.builddir, "images", "zImage")
+ dtb = os.path.join(self.builddir, "images", "vexpress-v2p-ca9.dtb")
+ self.emulator.boot(arch="armv7", kernel=kern,
+ kernel_cmdline=["console=ttyAMA0"],
+ options=["-M", "vexpress-a9", "-dtb", dtb, "-initrd", initrd])
+ self.emulator.login()
+ self.assertRunOk("true")
--
2.37.2
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot
^ permalink raw reply related [flat|nested] 5+ messages in thread
* Re: [Buildroot] [RFC next 1/2] linux: fix builds for kernels < 5.6 and host-gcc >= 10
2022-08-22 22:14 [Buildroot] [RFC next 1/2] linux: fix builds for kernels < 5.6 and host-gcc >= 10 Julien Olivain
2022-08-22 22:14 ` [Buildroot] [RFC next 2/2] support/testing/tests/core/test_kernel3_10: new test Julien Olivain
@ 2022-12-31 19:33 ` Yann E. MORIN
2023-01-04 13:04 ` Peter Korsgaard
1 sibling, 1 reply; 5+ messages in thread
From: Yann E. MORIN @ 2022-12-31 19:33 UTC (permalink / raw)
To: Julien Olivain; +Cc: buildroot
Julien, All,
On 2022-08-23 00:14 +0200, Julien Olivain spake thusly:
> During a linux-backports update, it was found that kernel v3.10.108 was
> failing to compile. See:
> https://lists.buildroot.org/pipermail/buildroot/2022-August/649507.html
>
> This issue was introduced by commit 9b41b54be07711c10ad13ce157be272ed1cf402e
> "linux: fix build with host-gcc 10+", which removes all declarations of
> "yylloc" symbols in the dtc parser. This symbol is generated by bison, if the
> "%locations" directive is provided in the parser. See:
> https://git.savannah.gnu.org/cgit/bison.git/tree/doc/bison.texi?h=v3.8.2#n5984
>
> Kernel versions < 5.6 did not include this directive, so removing all
> yylloc declararions in the parser also was failing for those version.
>
> In the kernel, dtc was updated to v1.5.1-22-gc40aeb60b47a in:
> https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=0cec114e36606412908a35695a5db944cec2e3db
> This commit is included in kernel v5.6.
>
> This dtc update include the dtc commit:
> https://git.kernel.org/pub/scm/utils/dtc/dtc.git/commit/?id=7150286225476345bd6e7312331e3baf4d621c32
> which adds the '%locations' directive.
>
> This commit fixes the issue by programmatically adding the '%locations'
> Bison directive, if it's not found in the parser file.
>
> Fixes:
> - https://bugs.busybox.net/show_bug.cgi?id=14971
>
> Signed-off-by: Julien Olivain <ju.o@free.fr>
I eventually took some time to look into that issue, and I can't think
of a better solution than what you proposed.
Applied to master, thanks.
Regards,
Yann E. MORIN.
> ---
> Tested by:
> - Building a recent kernel:
> qemu_aarch64_virt_defconfig (Kernel v5.15.18)
> - Building an old kernel:
> qemu_arm_vexpress_defconfig with kernel 3.10.108
> ---
> linux/linux.mk | 14 ++++++++++++++
> 1 file changed, 14 insertions(+)
>
> diff --git a/linux/linux.mk b/linux/linux.mk
> index 3d9ac37959..f3eb9c32cb 100644
> --- a/linux/linux.mk
> +++ b/linux/linux.mk
> @@ -155,6 +155,7 @@ LINUX_MAKE_FLAGS = \
> INSTALL_MOD_PATH=$(TARGET_DIR) \
> CROSS_COMPILE="$(TARGET_CROSS)" \
> WERROR=0 \
> + REGENERATE_PARSERS=1 \
> DEPMOD=$(HOST_DIR)/sbin/depmod
>
> ifeq ($(BR2_REPRODUCIBLE),y)
> @@ -286,6 +287,19 @@ define LINUX_DROP_YYLLOC
> endef
> LINUX_POST_PATCH_HOOKS += LINUX_DROP_YYLLOC
>
> +# Kernel version < 5.6 breaks if host-gcc version is >= 10 and
> +# 'yylloc' symbol is removed in previous hook, due to missing
> +# '%locations' bison directive in dtc-parser.y. See:
> +# https://bugs.busybox.net/show_bug.cgi?id=14971
> +define LINUX_ADD_DTC_LOCATIONS
> + $(Q)DTC_PARSER=$(@D)/scripts/dtc/dtc-parser.y; \
> + if test -e "$${DTC_PARSER}" \
> + && ! grep -Eq '^%locations$$' "$${DTC_PARSER}" ; then \
> + $(SED) '/^%{$$/i %locations' "$${DTC_PARSER}"; \
> + fi
> +endef
> +LINUX_POST_PATCH_HOOKS += LINUX_ADD_DTC_LOCATIONS
> +
> # Older linux kernels use deprecated perl constructs in timeconst.pl
> # that were removed for perl 5.22+ so it breaks on newer distributions
> # Try a dry-run patch to see if this applies, if it does go ahead
> --
> 2.37.2
>
> _______________________________________________
> buildroot mailing list
> buildroot@buildroot.org
> https://lists.buildroot.org/mailman/listinfo/buildroot
--
.-----------------.--------------------.------------------.--------------------.
| Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ |
| +33 561 099 427 `------------.-------: X AGAINST | \e/ There is no |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [Buildroot] [RFC next 2/2] support/testing/tests/core/test_kernel3_10: new test
2022-08-22 22:14 ` [Buildroot] [RFC next 2/2] support/testing/tests/core/test_kernel3_10: new test Julien Olivain
@ 2022-12-31 19:38 ` Yann E. MORIN
0 siblings, 0 replies; 5+ messages in thread
From: Yann E. MORIN @ 2022-12-31 19:38 UTC (permalink / raw)
To: Julien Olivain; +Cc: buildroot
Julien, All,
On 2022-08-23 00:14 +0200, Julien Olivain spake thusly:
> This commit add a test to check that a relatively old 3.10 kernel are
> building successfully in Buildroot. This test was written for
> investigating the bug described in:
> - https://bugs.busybox.net/show_bug.cgi?id=14971
>
> Signed-off-by: Julien Olivain <ju.o@free.fr>
> ---
[--SNIP--]
> diff --git a/support/testing/tests/core/test_kernel3_10.py b/support/testing/tests/core/test_kernel3_10.py
> new file mode 100644
> index 0000000000..5ff5ed1cad
> --- /dev/null
> +++ b/support/testing/tests/core/test_kernel3_10.py
> @@ -0,0 +1,43 @@
> +import os
> +
> +import infra.basetest
> +
> +
> +# For testing the compilation of an old 3.10 kernel, we use a armv7
> +# gcc5 bootlin toolchain, which also includes 3.10 kernel headers.
I'm not sure we want such a test. If we wanted to test that we can build
an older kernel, then we should use a state-of-the-art toolchain, i.e.
at least the default settings in Buildroot, not an older toolchain.
However, that would validate that he kernel itself can be built with a
recent compiler, not that the kernel build infra can be built; indeed,
the infra uses the host compiler, not the target compiler, but we have
no way to test with a latest-ish gcc:
* our reference image is still using an old system especially because
we also want to ensure that we can still use Buildroot on such old
systems;
* we can't know what users have on their machine if they run the
runtime test locally.
So, I am not sure this test is entirely meaningful.
Regards,
Yann E. MORIN.
> +class TestOldKernel3_10(infra.basetest.BRTest):
> + config = \
> + """
> + BR2_arm=y
> + BR2_cortex_a9=y
> + BR2_ARM_EABIHF=y
> + BR2_ARM_ENABLE_NEON=y
> + BR2_ARM_ENABLE_VFP=y
> + BR2_LINUX_KERNEL=y
> + BR2_LINUX_KERNEL_CUSTOM_VERSION=y
> + BR2_LINUX_KERNEL_CUSTOM_VERSION_VALUE="3.10.108"
> + BR2_LINUX_KERNEL_DEFCONFIG="vexpress"
> + BR2_LINUX_KERNEL_DTS_SUPPORT=y
> + BR2_LINUX_KERNEL_INTREE_DTS_NAME="vexpress-v2p-ca9"
> + BR2_TARGET_GENERIC_GETTY_PORT="ttyAMA0"
> + BR2_TARGET_ROOTFS_CPIO=y
> + BR2_TARGET_ROOTFS_CPIO_GZIP=y
> + BR2_TOOLCHAIN_EXTERNAL=y
> + BR2_TOOLCHAIN_EXTERNAL_CUSTOM=y
> + BR2_TOOLCHAIN_EXTERNAL_CUSTOM_GLIBC=y
> + BR2_TOOLCHAIN_EXTERNAL_CXX=y
> + BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y
> + BR2_TOOLCHAIN_EXTERNAL_GCC_5=y
> + BR2_TOOLCHAIN_EXTERNAL_HEADERS_3_10=y
> + BR2_TOOLCHAIN_EXTERNAL_URL="https://toolchains.bootlin.com/downloads/releases/toolchains/armv7-eabihf/tarballs/armv7-eabihf--glibc--stable-2017.05-toolchains-1-1.tar.bz2"
> + """
> +
> + def test_run(self):
> + initrd = os.path.join(self.builddir, "images", "rootfs.cpio.gz")
> + kern = os.path.join(self.builddir, "images", "zImage")
> + dtb = os.path.join(self.builddir, "images", "vexpress-v2p-ca9.dtb")
> + self.emulator.boot(arch="armv7", kernel=kern,
> + kernel_cmdline=["console=ttyAMA0"],
> + options=["-M", "vexpress-a9", "-dtb", dtb, "-initrd", initrd])
> + self.emulator.login()
> + self.assertRunOk("true")
> --
> 2.37.2
>
> _______________________________________________
> buildroot mailing list
> buildroot@buildroot.org
> https://lists.buildroot.org/mailman/listinfo/buildroot
--
.-----------------.--------------------.------------------.--------------------.
| Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ |
| +33 561 099 427 `------------.-------: X AGAINST | \e/ There is no |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [Buildroot] [RFC next 1/2] linux: fix builds for kernels < 5.6 and host-gcc >= 10
2022-12-31 19:33 ` [Buildroot] [RFC next 1/2] linux: fix builds for kernels < 5.6 and host-gcc >= 10 Yann E. MORIN
@ 2023-01-04 13:04 ` Peter Korsgaard
0 siblings, 0 replies; 5+ messages in thread
From: Peter Korsgaard @ 2023-01-04 13:04 UTC (permalink / raw)
To: Yann E. MORIN; +Cc: Julien Olivain, buildroot
>>>>> "Yann" == Yann E MORIN <yann.morin.1998@free.fr> writes:
> Julien, All,
> On 2022-08-23 00:14 +0200, Julien Olivain spake thusly:
>> During a linux-backports update, it was found that kernel v3.10.108 was
>> failing to compile. See:
>> https://lists.buildroot.org/pipermail/buildroot/2022-August/649507.html
>>
>> This issue was introduced by commit 9b41b54be07711c10ad13ce157be272ed1cf402e
>> "linux: fix build with host-gcc 10+", which removes all declarations of
>> "yylloc" symbols in the dtc parser. This symbol is generated by bison, if the
>> "%locations" directive is provided in the parser. See:
>> https://git.savannah.gnu.org/cgit/bison.git/tree/doc/bison.texi?h=v3.8.2#n5984
>>
>> Kernel versions < 5.6 did not include this directive, so removing all
>> yylloc declararions in the parser also was failing for those version.
>>
>> In the kernel, dtc was updated to v1.5.1-22-gc40aeb60b47a in:
>> https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=0cec114e36606412908a35695a5db944cec2e3db
>> This commit is included in kernel v5.6.
>>
>> This dtc update include the dtc commit:
>> https://git.kernel.org/pub/scm/utils/dtc/dtc.git/commit/?id=7150286225476345bd6e7312331e3baf4d621c32
>> which adds the '%locations' directive.
>>
>> This commit fixes the issue by programmatically adding the '%locations'
>> Bison directive, if it's not found in the parser file.
>>
>> Fixes:
>> - https://bugs.busybox.net/show_bug.cgi?id=14971
>>
>> Signed-off-by: Julien Olivain <ju.o@free.fr>
> I eventually took some time to look into that issue, and I can't think
> of a better solution than what you proposed.
> Applied to master, thanks.
Committed to 2022.11.x and 2022.02.x, thanks.
--
Bye, Peter Korsgaard
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2023-01-04 13:04 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-08-22 22:14 [Buildroot] [RFC next 1/2] linux: fix builds for kernels < 5.6 and host-gcc >= 10 Julien Olivain
2022-08-22 22:14 ` [Buildroot] [RFC next 2/2] support/testing/tests/core/test_kernel3_10: new test Julien Olivain
2022-12-31 19:38 ` Yann E. MORIN
2022-12-31 19:33 ` [Buildroot] [RFC next 1/2] linux: fix builds for kernels < 5.6 and host-gcc >= 10 Yann E. MORIN
2023-01-04 13:04 ` Peter Korsgaard
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.