[PATCH v2] kernel: relocate copy of module.lds to module compilation task

[PATCH v2] kernel: relocate copy of module.lds to module compilation task

[Bruce Ashfield]

From: Bruce Ashfield <bruce.ashfield@gmail.com>

There were two copies of this patch floating around, and the merged
variant has the copy in the wrong place.

module.lds is only created during modules_prepare, and that target is
not invoked during our main build of the kernel. We aren't about to
change the kernel build (there's no need), so we move the copy into
the compile_kernelmodules task. After that runs, we have module.lds
availble to copy.

This has been tested against clean kernel + out of tree module
builds, and the dependencies are correct that the file is copied
before the out of tree module build starts.

Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
---

v2:

I had done my test builds on a branch that didn't actually contain my
change, hence why 5.8 builds where blowing up.

I switched to the if [ ] model, since that doesn't run afoul of the
exit handler. I do find it curious that the compile task behaves so
differently than the install ones (even with set +e, I couldn't use
the [ ] method.

Anyway, this is tested against 5.8 and 5.10 + eternal module builds

Bruce

 meta/classes/kernel.bbclass | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
index ccd74e61e8..8b28ee626f 100644
--- a/meta/classes/kernel.bbclass
+++ b/meta/classes/kernel.bbclass
@@ -391,6 +391,10 @@ do_compile_kernelmodules() {
 		# other kernel modules and will look at this
 		# file to do symbol lookups
 		cp ${B}/Module.symvers ${STAGING_KERNEL_BUILDDIR}/
+		# 5.10+ kernels have module.lds that we need to copy for external module builds
+		if [ -e "${B}/scripts/module.lds" ]; then
+			install -Dm 0644 ${B}/scripts/module.lds ${STAGING_KERNEL_BUILDDIR}/scripts/module.lds
+		fi
 	else
 		bbnote "no modules to compile"
 	fi
@@ -494,7 +498,6 @@ do_shared_workdir () {
 	# Copy files required for module builds
 	cp System.map $kerneldir/System.map-${KERNEL_VERSION}
 	[ -e Module.symvers ] && cp Module.symvers $kerneldir/
-	[ -e scripts/module.lds ] && install -Dm 0644 scripts/module.lds $kerneldir/scripts/module.lds
 	cp .config $kerneldir/
 	mkdir -p $kerneldir/include/config
 	cp include/config/kernel.release $kerneldir/include/config/kernel.release
-- 
2.19.1

Re: [OE-core] [PATCH v2] kernel: relocate copy of module.lds to module compilation task

[Khem Raj]

thanks this version looks good and works too.

On Tue, Nov 17, 2020 at 8:56 PM Bruce Ashfield <bruce.ashfield@gmail.com> wrote:
>
> From: Bruce Ashfield <bruce.ashfield@gmail.com>
>
> There were two copies of this patch floating around, and the merged
> variant has the copy in the wrong place.
>
> module.lds is only created during modules_prepare, and that target is
> not invoked during our main build of the kernel. We aren't about to
> change the kernel build (there's no need), so we move the copy into
> the compile_kernelmodules task. After that runs, we have module.lds
> availble to copy.
>
> This has been tested against clean kernel + out of tree module
> builds, and the dependencies are correct that the file is copied
> before the out of tree module build starts.
>
> Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
> ---
>
> v2:
>
> I had done my test builds on a branch that didn't actually contain my
> change, hence why 5.8 builds where blowing up.
>
> I switched to the if [ ] model, since that doesn't run afoul of the
> exit handler. I do find it curious that the compile task behaves so
> differently than the install ones (even with set +e, I couldn't use
> the [ ] method.
>
> Anyway, this is tested against 5.8 and 5.10 + eternal module builds
>
> Bruce
>
>  meta/classes/kernel.bbclass | 5 ++++-
>  1 file changed, 4 insertions(+), 1 deletion(-)
>
> diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
> index ccd74e61e8..8b28ee626f 100644
> --- a/meta/classes/kernel.bbclass
> +++ b/meta/classes/kernel.bbclass
> @@ -391,6 +391,10 @@ do_compile_kernelmodules() {
>                 # other kernel modules and will look at this
>                 # file to do symbol lookups
>                 cp ${B}/Module.symvers ${STAGING_KERNEL_BUILDDIR}/
> +               # 5.10+ kernels have module.lds that we need to copy for external module builds
> +               if [ -e "${B}/scripts/module.lds" ]; then
> +                       install -Dm 0644 ${B}/scripts/module.lds ${STAGING_KERNEL_BUILDDIR}/scripts/module.lds
> +               fi
>         else
>                 bbnote "no modules to compile"
>         fi
> @@ -494,7 +498,6 @@ do_shared_workdir () {
>         # Copy files required for module builds
>         cp System.map $kerneldir/System.map-${KERNEL_VERSION}
>         [ -e Module.symvers ] && cp Module.symvers $kerneldir/
> -       [ -e scripts/module.lds ] && install -Dm 0644 scripts/module.lds $kerneldir/scripts/module.lds
>         cp .config $kerneldir/
>         mkdir -p $kerneldir/include/config
>         cp include/config/kernel.release $kerneldir/include/config/kernel.release
> --
> 2.19.1
>
>
> 
>

Re: [OE-core] [PATCH v2] kernel: relocate copy of module.lds to module compilation task

[Nicolas Dechesne]

hi Steve and Bruce,

On Wed, Nov 18, 2020 at 5:56 AM Bruce Ashfield <bruce.ashfield@gmail.com> wrote:
>
> From: Bruce Ashfield <bruce.ashfield@gmail.com>
>
> There were two copies of this patch floating around, and the merged
> variant has the copy in the wrong place.
>
> module.lds is only created during modules_prepare, and that target is
> not invoked during our main build of the kernel. We aren't about to
> change the kernel build (there's no need), so we move the copy into
> the compile_kernelmodules task. After that runs, we have module.lds
> availble to copy.
>
> This has been tested against clean kernel + out of tree module
> builds, and the dependencies are correct that the file is copied
> before the out of tree module build starts.
>
> Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>


Can we get this patch into dunfell? This is fixing a bug for BSP using
5.10+ kernel, so I believe this is important for LTS. I haven't
attempted to backport it yet.. but let me know if you need help with
the backport.

> ---
>
> v2:
>
> I had done my test builds on a branch that didn't actually contain my
> change, hence why 5.8 builds where blowing up.
>
> I switched to the if [ ] model, since that doesn't run afoul of the
> exit handler. I do find it curious that the compile task behaves so
> differently than the install ones (even with set +e, I couldn't use
> the [ ] method.
>
> Anyway, this is tested against 5.8 and 5.10 + eternal module builds
>
> Bruce
>
>  meta/classes/kernel.bbclass | 5 ++++-
>  1 file changed, 4 insertions(+), 1 deletion(-)
>
> diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
> index ccd74e61e8..8b28ee626f 100644
> --- a/meta/classes/kernel.bbclass
> +++ b/meta/classes/kernel.bbclass
> @@ -391,6 +391,10 @@ do_compile_kernelmodules() {
>                 # other kernel modules and will look at this
>                 # file to do symbol lookups
>                 cp ${B}/Module.symvers ${STAGING_KERNEL_BUILDDIR}/
> +               # 5.10+ kernels have module.lds that we need to copy for external module builds
> +               if [ -e "${B}/scripts/module.lds" ]; then
> +                       install -Dm 0644 ${B}/scripts/module.lds ${STAGING_KERNEL_BUILDDIR}/scripts/module.lds
> +               fi
>         else
>                 bbnote "no modules to compile"
>         fi
> @@ -494,7 +498,6 @@ do_shared_workdir () {
>         # Copy files required for module builds
>         cp System.map $kerneldir/System.map-${KERNEL_VERSION}
>         [ -e Module.symvers ] && cp Module.symvers $kerneldir/
> -       [ -e scripts/module.lds ] && install -Dm 0644 scripts/module.lds $kerneldir/scripts/module.lds
>         cp .config $kerneldir/
>         mkdir -p $kerneldir/include/config
>         cp include/config/kernel.release $kerneldir/include/config/kernel.release
> --
> 2.19.1
>
>
> 
>

Re: [OE-core] [PATCH v2] kernel: relocate copy of module.lds to module compilation task

[Bruce Ashfield]

On Mon, Nov 30, 2020 at 2:38 AM Nicolas Dechesne
<nicolas.dechesne@linaro.org> wrote:
>
> hi Steve and Bruce,
>
> On Wed, Nov 18, 2020 at 5:56 AM Bruce Ashfield <bruce.ashfield@gmail.com> wrote:
> >
> > From: Bruce Ashfield <bruce.ashfield@gmail.com>
> >
> > There were two copies of this patch floating around, and the merged
> > variant has the copy in the wrong place.
> >
> > module.lds is only created during modules_prepare, and that target is
> > not invoked during our main build of the kernel. We aren't about to
> > change the kernel build (there's no need), so we move the copy into
> > the compile_kernelmodules task. After that runs, we have module.lds
> > availble to copy.
> >
> > This has been tested against clean kernel + out of tree module
> > builds, and the dependencies are correct that the file is copied
> > before the out of tree module build starts.
> >
> > Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
>
>
> Can we get this patch into dunfell? This is fixing a bug for BSP using
> 5.10+ kernel, so I believe this is important for LTS. I haven't
> attempted to backport it yet.. but let me know if you need help with
> the backport.

For a backport, you need this patch and also the one that it is tweaking:
[kernel: provide module.lds for out of tree builds in v5.10+], but I didn't
check to see if Steve had already grabbed that one.

I also don't see any issue with this, and the backport is simple enough since
it is additive to what we copy into the artifacts.

Like building on newer distros, we will eventually run into newer kernel
support that we can't backport or support on dunfell .. but this isn't that
case!

Steve: it should be a simple cherry pick, if it isn't, let me know and I can
send the patches.

Cheers,

Bruce

>
>
> > ---
> >
> > v2:
> >
> > I had done my test builds on a branch that didn't actually contain my
> > change, hence why 5.8 builds where blowing up.
> >
> > I switched to the if [ ] model, since that doesn't run afoul of the
> > exit handler. I do find it curious that the compile task behaves so
> > differently than the install ones (even with set +e, I couldn't use
> > the [ ] method.
> >
> > Anyway, this is tested against 5.8 and 5.10 + eternal module builds
> >
> > Bruce
> >
> >  meta/classes/kernel.bbclass | 5 ++++-
> >  1 file changed, 4 insertions(+), 1 deletion(-)
> >
> > diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
> > index ccd74e61e8..8b28ee626f 100644
> > --- a/meta/classes/kernel.bbclass
> > +++ b/meta/classes/kernel.bbclass
> > @@ -391,6 +391,10 @@ do_compile_kernelmodules() {
> >                 # other kernel modules and will look at this
> >                 # file to do symbol lookups
> >                 cp ${B}/Module.symvers ${STAGING_KERNEL_BUILDDIR}/
> > +               # 5.10+ kernels have module.lds that we need to copy for external module builds
> > +               if [ -e "${B}/scripts/module.lds" ]; then
> > +                       install -Dm 0644 ${B}/scripts/module.lds ${STAGING_KERNEL_BUILDDIR}/scripts/module.lds
> > +               fi
> >         else
> >                 bbnote "no modules to compile"
> >         fi
> > @@ -494,7 +498,6 @@ do_shared_workdir () {
> >         # Copy files required for module builds
> >         cp System.map $kerneldir/System.map-${KERNEL_VERSION}
> >         [ -e Module.symvers ] && cp Module.symvers $kerneldir/
> > -       [ -e scripts/module.lds ] && install -Dm 0644 scripts/module.lds $kerneldir/scripts/module.lds
> >         cp .config $kerneldir/
> >         mkdir -p $kerneldir/include/config
> >         cp include/config/kernel.release $kerneldir/include/config/kernel.release
> > --
> > 2.19.1
> >
> >
> > 
> >



-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

Re: [OE-core] [PATCH v2] kernel: relocate copy of module.lds to module compilation task

[Steve Sakoman]

On Mon, Nov 30, 2020 at 3:24 AM Bruce Ashfield <bruce.ashfield@gmail.com> wrote:
>
> On Mon, Nov 30, 2020 at 2:38 AM Nicolas Dechesne
> <nicolas.dechesne@linaro.org> wrote:
> >
> > hi Steve and Bruce,
> >
> > On Wed, Nov 18, 2020 at 5:56 AM Bruce Ashfield <bruce.ashfield@gmail.com> wrote:
> > >
> > > From: Bruce Ashfield <bruce.ashfield@gmail.com>
> > >
> > > There were two copies of this patch floating around, and the merged
> > > variant has the copy in the wrong place.
> > >
> > > module.lds is only created during modules_prepare, and that target is
> > > not invoked during our main build of the kernel. We aren't about to
> > > change the kernel build (there's no need), so we move the copy into
> > > the compile_kernelmodules task. After that runs, we have module.lds
> > > availble to copy.
> > >
> > > This has been tested against clean kernel + out of tree module
> > > builds, and the dependencies are correct that the file is copied
> > > before the out of tree module build starts.
> > >
> > > Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
> >
> >
> > Can we get this patch into dunfell? This is fixing a bug for BSP using
> > 5.10+ kernel, so I believe this is important for LTS.

Certainly!

> For a backport, you need this patch and also the one that it is tweaking:
> [kernel: provide module.lds for out of tree builds in v5.10+], but I didn't
> check to see if Steve had already grabbed that one.
>
> I also don't see any issue with this, and the backport is simple enough since
> it is additive to what we copy into the artifacts.
>
> Like building on newer distros, we will eventually run into newer kernel
> support that we can't backport or support on dunfell .. but this isn't that
> case!
>
> Steve: it should be a simple cherry pick, if it isn't, let me know and I can
> send the patches.

A simple cherry-pick was successful.  Will start testing later this morning.

Steve

> > >
> > > v2:
> > >
> > > I had done my test builds on a branch that didn't actually contain my
> > > change, hence why 5.8 builds where blowing up.
> > >
> > > I switched to the if [ ] model, since that doesn't run afoul of the
> > > exit handler. I do find it curious that the compile task behaves so
> > > differently than the install ones (even with set +e, I couldn't use
> > > the [ ] method.
> > >
> > > Anyway, this is tested against 5.8 and 5.10 + eternal module builds
> > >
> > > Bruce
> > >
> > >  meta/classes/kernel.bbclass | 5 ++++-
> > >  1 file changed, 4 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
> > > index ccd74e61e8..8b28ee626f 100644
> > > --- a/meta/classes/kernel.bbclass
> > > +++ b/meta/classes/kernel.bbclass
> > > @@ -391,6 +391,10 @@ do_compile_kernelmodules() {
> > >                 # other kernel modules and will look at this
> > >                 # file to do symbol lookups
> > >                 cp ${B}/Module.symvers ${STAGING_KERNEL_BUILDDIR}/
> > > +               # 5.10+ kernels have module.lds that we need to copy for external module builds
> > > +               if [ -e "${B}/scripts/module.lds" ]; then
> > > +                       install -Dm 0644 ${B}/scripts/module.lds ${STAGING_KERNEL_BUILDDIR}/scripts/module.lds
> > > +               fi
> > >         else
> > >                 bbnote "no modules to compile"
> > >         fi
> > > @@ -494,7 +498,6 @@ do_shared_workdir () {
> > >         # Copy files required for module builds
> > >         cp System.map $kerneldir/System.map-${KERNEL_VERSION}
> > >         [ -e Module.symvers ] && cp Module.symvers $kerneldir/
> > > -       [ -e scripts/module.lds ] && install -Dm 0644 scripts/module.lds $kerneldir/scripts/module.lds
> > >         cp .config $kerneldir/
> > >         mkdir -p $kerneldir/include/config
> > >         cp include/config/kernel.release $kerneldir/include/config/kernel.release
> > > --
> > > 2.19.1
> > >
> > >
> > > 
> > >
>
>
>
> --
> - Thou shalt not follow the NULL pointer, for chaos and madness await
> thee at its end
> - "Use the force Harry" - Gandalf, Star Trek II

Re: [OE-core] [PATCH v2] kernel: relocate copy of module.lds to module compilation task

[Steve Sakoman]

On Sun, Nov 29, 2020 at 9:38 PM Nicolas Dechesne
<nicolas.dechesne@linaro.org> wrote:
>
> hi Steve and Bruce,
>
> On Wed, Nov 18, 2020 at 5:56 AM Bruce Ashfield <bruce.ashfield@gmail.com> wrote:
> >
> > From: Bruce Ashfield <bruce.ashfield@gmail.com>
> >
> > There were two copies of this patch floating around, and the merged
> > variant has the copy in the wrong place.
> >
> > module.lds is only created during modules_prepare, and that target is
> > not invoked during our main build of the kernel. We aren't about to
> > change the kernel build (there's no need), so we move the copy into
> > the compile_kernelmodules task. After that runs, we have module.lds
> > availble to copy.
> >
> > This has been tested against clean kernel + out of tree module
> > builds, and the dependencies are correct that the file is copied
> > before the out of tree module build starts.
> >
> > Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
>
>
> Can we get this patch into dunfell? This is fixing a bug for BSP using
> 5.10+ kernel, so I believe this is important for LTS. I haven't
> attempted to backport it yet.. but let me know if you need help with
> the backport.

Is this the only patch you need to fix the 5.10+ kernel issue in dunfell?

In general I haven't been taking patches that claim to be for kernels
beyond 5.4, so I suspect there may be a handful of other patches you
might need.

Steve

> > I had done my test builds on a branch that didn't actually contain my
> > change, hence why 5.8 builds where blowing up.
> >
> > I switched to the if [ ] model, since that doesn't run afoul of the
> > exit handler. I do find it curious that the compile task behaves so
> > differently than the install ones (even with set +e, I couldn't use
> > the [ ] method.
> >
> > Anyway, this is tested against 5.8 and 5.10 + eternal module builds
> >
> > Bruce
> >
> >  meta/classes/kernel.bbclass | 5 ++++-
> >  1 file changed, 4 insertions(+), 1 deletion(-)
> >
> > diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
> > index ccd74e61e8..8b28ee626f 100644
> > --- a/meta/classes/kernel.bbclass
> > +++ b/meta/classes/kernel.bbclass
> > @@ -391,6 +391,10 @@ do_compile_kernelmodules() {
> >                 # other kernel modules and will look at this
> >                 # file to do symbol lookups
> >                 cp ${B}/Module.symvers ${STAGING_KERNEL_BUILDDIR}/
> > +               # 5.10+ kernels have module.lds that we need to copy for external module builds
> > +               if [ -e "${B}/scripts/module.lds" ]; then
> > +                       install -Dm 0644 ${B}/scripts/module.lds ${STAGING_KERNEL_BUILDDIR}/scripts/module.lds
> > +               fi
> >         else
> >                 bbnote "no modules to compile"
> >         fi
> > @@ -494,7 +498,6 @@ do_shared_workdir () {
> >         # Copy files required for module builds
> >         cp System.map $kerneldir/System.map-${KERNEL_VERSION}
> >         [ -e Module.symvers ] && cp Module.symvers $kerneldir/
> > -       [ -e scripts/module.lds ] && install -Dm 0644 scripts/module.lds $kerneldir/scripts/module.lds
> >         cp .config $kerneldir/
> >         mkdir -p $kerneldir/include/config
> >         cp include/config/kernel.release $kerneldir/include/config/kernel.release
> > --
> > 2.19.1
> >
> >
> > 
> >

Re: [OE-core] [PATCH v2] kernel: relocate copy of module.lds to module compilation task

[Nicolas Dechesne]

On Mon, Nov 30, 2020 at 4:34 PM Steve Sakoman <steve@sakoman.com> wrote:
>
> On Sun, Nov 29, 2020 at 9:38 PM Nicolas Dechesne
> <nicolas.dechesne@linaro.org> wrote:
> >
> > hi Steve and Bruce,
> >
> > On Wed, Nov 18, 2020 at 5:56 AM Bruce Ashfield <bruce.ashfield@gmail.com> wrote:
> > >
> > > From: Bruce Ashfield <bruce.ashfield@gmail.com>
> > >
> > > There were two copies of this patch floating around, and the merged
> > > variant has the copy in the wrong place.
> > >
> > > module.lds is only created during modules_prepare, and that target is
> > > not invoked during our main build of the kernel. We aren't about to
> > > change the kernel build (there's no need), so we move the copy into
> > > the compile_kernelmodules task. After that runs, we have module.lds
> > > availble to copy.
> > >
> > > This has been tested against clean kernel + out of tree module
> > > builds, and the dependencies are correct that the file is copied
> > > before the out of tree module build starts.
> > >
> > > Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
> >
> >
> > Can we get this patch into dunfell? This is fixing a bug for BSP using
> > 5.10+ kernel, so I believe this is important for LTS. I haven't
> > attempted to backport it yet.. but let me know if you need help with
> > the backport.
>
> Is this the only patch you need to fix the 5.10+ kernel issue in dunfell?
>
> In general I haven't been taking patches that claim to be for kernels
> beyond 5.4, so I suspect there may be a handful of other patches you
> might need.

Not that I am aware! Thanks for picking it!

>
> Steve
>
> > > I had done my test builds on a branch that didn't actually contain my
> > > change, hence why 5.8 builds where blowing up.
> > >
> > > I switched to the if [ ] model, since that doesn't run afoul of the
> > > exit handler. I do find it curious that the compile task behaves so
> > > differently than the install ones (even with set +e, I couldn't use
> > > the [ ] method.
> > >
> > > Anyway, this is tested against 5.8 and 5.10 + eternal module builds
> > >
> > > Bruce
> > >
> > >  meta/classes/kernel.bbclass | 5 ++++-
> > >  1 file changed, 4 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
> > > index ccd74e61e8..8b28ee626f 100644
> > > --- a/meta/classes/kernel.bbclass
> > > +++ b/meta/classes/kernel.bbclass
> > > @@ -391,6 +391,10 @@ do_compile_kernelmodules() {
> > >                 # other kernel modules and will look at this
> > >                 # file to do symbol lookups
> > >                 cp ${B}/Module.symvers ${STAGING_KERNEL_BUILDDIR}/
> > > +               # 5.10+ kernels have module.lds that we need to copy for external module builds
> > > +               if [ -e "${B}/scripts/module.lds" ]; then
> > > +                       install -Dm 0644 ${B}/scripts/module.lds ${STAGING_KERNEL_BUILDDIR}/scripts/module.lds
> > > +               fi
> > >         else
> > >                 bbnote "no modules to compile"
> > >         fi
> > > @@ -494,7 +498,6 @@ do_shared_workdir () {
> > >         # Copy files required for module builds
> > >         cp System.map $kerneldir/System.map-${KERNEL_VERSION}
> > >         [ -e Module.symvers ] && cp Module.symvers $kerneldir/
> > > -       [ -e scripts/module.lds ] && install -Dm 0644 scripts/module.lds $kerneldir/scripts/module.lds
> > >         cp .config $kerneldir/
> > >         mkdir -p $kerneldir/include/config
> > >         cp include/config/kernel.release $kerneldir/include/config/kernel.release
> > > --
> > > 2.19.1
> > >
> > >
> > > 
> > >