[PATCH] module.bbclass: swap AR and LD order

[PATCH] module.bbclass: swap AR and LD order

[Jason Wessel]

The 5.x kernels seem to have made a change to the linker command line processing.

When trying to build out of tree kernel modules, such as the
virtualbox guest additions, the following error is printed:

| make[1]: Entering directory '/opt/build-x86-64/tmp/work/qemux86_64-poky-linux/vboxguestdrivers/6.1.2-r0/vbox_module/vboxvideo'
| make[3]: Entering directory '/opt/build-x86-64/tmp/work-shared/qemux86-64/kernel-build-artifacts'
| make V= CONFIG_MODULE_SIG= CONFIG_MODULE_SIG_ALL= -C /opt/build-x86-64/tmp/work/qemux86_64-poky-linux/vboxguestdrivers/6.1.2-r0/5.2.28-yocto-standard/build M=/opt/build-x86-64/tmp/work/qemux86_64-poky-linux/vboxguestdrivers/6.1.2-r0/vbox_module/vboxguest SRCROOT=/opt/build-x86-64/tmp/work/qemux86_64-poky-linux/vboxguestdrivers/6.1.2-r0/vbox_module/vboxguest modules
| make[3]: Entering directory '/opt/build-x86-64/tmp/work-shared/qemux86-64/kernel-build-artifacts'
|   LD [M]  /opt/build-x86-64/tmp/work/qemux86_64-poky-linux/vboxguestdrivers/6.1.2-r0/vbox_module/vboxvideo/vboxvideo.o
| x86_64-poky-linux-ld.bfd: cannot find AR=x86_64-poky-linux-ar: No such file or directory

The AR arguments passed to the make command are being processed as
command line arguments to the linker due to some changes in the
kernel's Makefile.  For now the easiest fix is to just swap the
arguments in the module.bbclass.

Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
---
 meta/classes/module.bbclass | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/meta/classes/module.bbclass b/meta/classes/module.bbclass
index c0dfa35061..e1e5fdaa72 100644
--- a/meta/classes/module.bbclass
+++ b/meta/classes/module.bbclass
@@ -38,8 +38,7 @@ module_do_compile() {
 	unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS
 	oe_runmake KERNEL_PATH=${STAGING_KERNEL_DIR}   \
 		   KERNEL_VERSION=${KERNEL_VERSION}    \
-		   CC="${KERNEL_CC}" LD="${KERNEL_LD}" \
-		   AR="${KERNEL_AR}" \
+		   CC="${KERNEL_CC}" AR="${KERNEL_AR}" LD="${KERNEL_LD}" \
 	           O=${STAGING_KERNEL_BUILDDIR} \
 		   KBUILD_EXTRA_SYMBOLS="${KBUILD_EXTRA_SYMBOLS}" \
 		   ${MAKE_TARGETS}
-- 
2.23.0

Re: [PATCH] module.bbclass: swap AR and LD order

[Christopher Larson]

[-- Attachment #1: Type: text/plain, Size: 2907 bytes --]

What makefile change caused this? That behavior doesn't make much sense
given how make processes its command-line arguments.

On Thu, Jan 23, 2020 at 3:34 PM Jason Wessel <jason.wessel@windriver.com>
wrote:

> The 5.x kernels seem to have made a change to the linker command line
> processing.
>
> When trying to build out of tree kernel modules, such as the
> virtualbox guest additions, the following error is printed:
>
> | make[1]: Entering directory
> '/opt/build-x86-64/tmp/work/qemux86_64-poky-linux/vboxguestdrivers/6.1.2-r0/vbox_module/vboxvideo'
> | make[3]: Entering directory
> '/opt/build-x86-64/tmp/work-shared/qemux86-64/kernel-build-artifacts'
> | make V= CONFIG_MODULE_SIG= CONFIG_MODULE_SIG_ALL= -C
> /opt/build-x86-64/tmp/work/qemux86_64-poky-linux/vboxguestdrivers/6.1.2-r0/5.2.28-yocto-standard/build
> M=/opt/build-x86-64/tmp/work/qemux86_64-poky-linux/vboxguestdrivers/6.1.2-r0/vbox_module/vboxguest
> SRCROOT=/opt/build-x86-64/tmp/work/qemux86_64-poky-linux/vboxguestdrivers/6.1.2-r0/vbox_module/vboxguest
> modules
> | make[3]: Entering directory
> '/opt/build-x86-64/tmp/work-shared/qemux86-64/kernel-build-artifacts'
> |   LD [M]
> /opt/build-x86-64/tmp/work/qemux86_64-poky-linux/vboxguestdrivers/6.1.2-r0/vbox_module/vboxvideo/vboxvideo.o
> | x86_64-poky-linux-ld.bfd: cannot find AR=x86_64-poky-linux-ar: No such
> file or directory
>
> The AR arguments passed to the make command are being processed as
> command line arguments to the linker due to some changes in the
> kernel's Makefile.  For now the easiest fix is to just swap the
> arguments in the module.bbclass.
>
> Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
> ---
>  meta/classes/module.bbclass | 3 +--
>  1 file changed, 1 insertion(+), 2 deletions(-)
>
> diff --git a/meta/classes/module.bbclass b/meta/classes/module.bbclass
> index c0dfa35061..e1e5fdaa72 100644
> --- a/meta/classes/module.bbclass
> +++ b/meta/classes/module.bbclass
> @@ -38,8 +38,7 @@ module_do_compile() {
>         unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS
>         oe_runmake KERNEL_PATH=${STAGING_KERNEL_DIR}   \
>                    KERNEL_VERSION=${KERNEL_VERSION}    \
> -                  CC="${KERNEL_CC}" LD="${KERNEL_LD}" \
> -                  AR="${KERNEL_AR}" \
> +                  CC="${KERNEL_CC}" AR="${KERNEL_AR}" LD="${KERNEL_LD}" \
>                    O=${STAGING_KERNEL_BUILDDIR} \
>                    KBUILD_EXTRA_SYMBOLS="${KBUILD_EXTRA_SYMBOLS}" \
>                    ${MAKE_TARGETS}
> --
> 2.23.0
>
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>


-- 
Christopher Larson
kergoth at gmail dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Senior Software Engineer, Mentor Graphics

[-- Attachment #2: Type: text/html, Size: 3829 bytes --]

Re: [PATCH] module.bbclass: swap AR and LD order

[Khem Raj]

On 1/24/20 10:19 AM, Christopher Larson wrote:
> What makefile change caused this? That behavior doesn't make much sense 
> given how make processes its command-line arguments.
> 

I agree with you here, it could be a rare check where one want to define 
what collect progam should be used ( ld or ar )

besides, recently we moved AR to be gcc-ar/llvm-ar by default in config 
metadata, which is not going to work out of box for compiling kernel and 
modules, so overriding it with KERNEL_AR in 
module_do_compile/install/configure tasks is good change.

> On Thu, Jan 23, 2020 at 3:34 PM Jason Wessel <jason.wessel@windriver.com 
> <mailto:jason.wessel@windriver.com>> wrote:
> 
>     The 5.x kernels seem to have made a change to the linker command
>     line processing.
> 
>     When trying to build out of tree kernel modules, such as the
>     virtualbox guest additions, the following error is printed:
> 
>     | make[1]: Entering directory
>     '/opt/build-x86-64/tmp/work/qemux86_64-poky-linux/vboxguestdrivers/6.1.2-r0/vbox_module/vboxvideo'
>     | make[3]: Entering directory
>     '/opt/build-x86-64/tmp/work-shared/qemux86-64/kernel-build-artifacts'
>     | make V= CONFIG_MODULE_SIG= CONFIG_MODULE_SIG_ALL= -C
>     /opt/build-x86-64/tmp/work/qemux86_64-poky-linux/vboxguestdrivers/6.1.2-r0/5.2.28-yocto-standard/build
>     M=/opt/build-x86-64/tmp/work/qemux86_64-poky-linux/vboxguestdrivers/6.1.2-r0/vbox_module/vboxguest
>     SRCROOT=/opt/build-x86-64/tmp/work/qemux86_64-poky-linux/vboxguestdrivers/6.1.2-r0/vbox_module/vboxguest
>     modules
>     | make[3]: Entering directory
>     '/opt/build-x86-64/tmp/work-shared/qemux86-64/kernel-build-artifacts'
>     |   LD [M] 
>     /opt/build-x86-64/tmp/work/qemux86_64-poky-linux/vboxguestdrivers/6.1.2-r0/vbox_module/vboxvideo/vboxvideo.o
>     | x86_64-poky-linux-ld.bfd: cannot find AR=x86_64-poky-linux-ar: No
>     such file or directory
> 
>     The AR arguments passed to the make command are being processed as
>     command line arguments to the linker due to some changes in the
>     kernel's Makefile.  For now the easiest fix is to just swap the
>     arguments in the module.bbclass.
> 
>     Signed-off-by: Jason Wessel <jason.wessel@windriver.com
>     <mailto:jason.wessel@windriver.com>>
>     ---
>       meta/classes/module.bbclass | 3 +--
>       1 file changed, 1 insertion(+), 2 deletions(-)
> 
>     diff --git a/meta/classes/module.bbclass b/meta/classes/module.bbclass
>     index c0dfa35061..e1e5fdaa72 100644
>     --- a/meta/classes/module.bbclass
>     +++ b/meta/classes/module.bbclass
>     @@ -38,8 +38,7 @@ module_do_compile() {
>              unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS
>              oe_runmake KERNEL_PATH=${STAGING_KERNEL_DIR}   \
>                         KERNEL_VERSION=${KERNEL_VERSION}    \
>     -                  CC="${KERNEL_CC}" LD="${KERNEL_LD}" \
>     -                  AR="${KERNEL_AR}" \
>     +                  CC="${KERNEL_CC}" AR="${KERNEL_AR}"
>     LD="${KERNEL_LD}" \
>                         O=${STAGING_KERNEL_BUILDDIR} \
>                         KBUILD_EXTRA_SYMBOLS="${KBUILD_EXTRA_SYMBOLS}" \
>                         ${MAKE_TARGETS}
>     -- 
>     2.23.0
> 
>     -- 
>     _______________________________________________
>     Openembedded-core mailing list
>     Openembedded-core@lists.openembedded.org
>     <mailto:Openembedded-core@lists.openembedded.org>
>     http://lists.openembedded.org/mailman/listinfo/openembedded-core
> 
> 
> 
> -- 
> Christopher Larson
> kergoth at gmail dot com
> Founder - BitBake, OpenEmbedded, OpenZaurus
> Senior Software Engineer, Mentor Graphics
> 

Re: [PATCH] module.bbclass: swap AR and LD order

[Jason Wessel]

On 1/24/20 1:13 PM, Khem Raj wrote:
> On 1/24/20 10:19 AM, Christopher Larson wrote:
>> What makefile change caused this? That behavior doesn't make much sense 
>> given how make processes its command-line arguments.
>>
> 
> I agree with you here, it could be a rare check where one want to define 
> what collect progam should be used ( ld or ar )
> 
> besides, recently we moved AR to be gcc-ar/llvm-ar by default in config 
> metadata, which is not going to work out of box for compiling kernel and 
> modules, so overriding it with KERNEL_AR in 
> module_do_compile/install/configure tasks is good change.

I am not actually sure which part of kbuild is parsing the LD
arguments.  My best guess is that it is actually all the nested
gnumake filter calls causing the issue.  

With some further experimentation I determined that a space at the end
of the LD= is what causes it to think the next argument is part of the
prior one.  The poky distro is defining KERNEL_LD as:

% bitbake vboxguestdrivers -e |grep ^KERNEL_LD
KERNEL_LD="x86_64-poky-linux-ld.bfd "

The kbuild really doesn't like that space at the end.  You can trash out
a host native kernel compile module build by adding the arguments:
    LD="ld " JUNK_VAR="junk"

Attempting to debug/fix kbuild doesn't seem like the right answer. 

The next version of this patch looks like what is below.  Would that be ok? 


diff --git a/meta/classes/module.bbclass b/meta/classes/module.bbclass
index c0dfa35061..7b0a85a06d 100644
--- a/meta/classes/module.bbclass
+++ b/meta/classes/module.bbclass
@@ -12,6 +12,9 @@ python __anonymous () {
         if dep.startswith("kernel-module-"):
             extra_symbols.append("${STAGING_INCDIR}/" + dep + "/Module.symvers")
     d.setVar('KBUILD_EXTRA_SYMBOLS', " ".join(extra_symbols))
+    d.setVar('KERNEL_CC', d.getVar('KERNEL_CC').rstrip())
+    d.setVar('KERNEL_LD', d.getVar('KERNEL_LD').rstrip())
+    d.setVar('KERNEL_AR', d.getVar('KERNEL_AR').rstrip())
 }

Re: [PATCH] module.bbclass: swap AR and LD order

[Khem Raj]

On Fri, Jan 24, 2020 at 1:05 PM Jason Wessel <jason.wessel@windriver.com> wrote:
>
> On 1/24/20 1:13 PM, Khem Raj wrote:
> > On 1/24/20 10:19 AM, Christopher Larson wrote:
> >> What makefile change caused this? That behavior doesn't make much sense
> >> given how make processes its command-line arguments.
> >>
> >
> > I agree with you here, it could be a rare check where one want to define
> > what collect progam should be used ( ld or ar )
> >
> > besides, recently we moved AR to be gcc-ar/llvm-ar by default in config
> > metadata, which is not going to work out of box for compiling kernel and
> > modules, so overriding it with KERNEL_AR in
> > module_do_compile/install/configure tasks is good change.
>
> I am not actually sure which part of kbuild is parsing the LD
> arguments.  My best guess is that it is actually all the nested
> gnumake filter calls causing the issue.
>
> With some further experimentation I determined that a space at the end
> of the LD= is what causes it to think the next argument is part of the
> prior one.  The poky distro is defining KERNEL_LD as:
>
> % bitbake vboxguestdrivers -e |grep ^KERNEL_LD
> KERNEL_LD="x86_64-poky-linux-ld.bfd "
>
> The kbuild really doesn't like that space at the end.  You can trash out
> a host native kernel compile module build by adding the arguments:
>     LD="ld " JUNK_VAR="junk"
>
> Attempting to debug/fix kbuild doesn't seem like the right answer.
>
> The next version of this patch looks like what is below.  Would that be ok?
>
>
> diff --git a/meta/classes/module.bbclass b/meta/classes/module.bbclass
> index c0dfa35061..7b0a85a06d 100644
> --- a/meta/classes/module.bbclass
> +++ b/meta/classes/module.bbclass
> @@ -12,6 +12,9 @@ python __anonymous () {
>          if dep.startswith("kernel-module-"):
>              extra_symbols.append("${STAGING_INCDIR}/" + dep + "/Module.symvers")
>      d.setVar('KBUILD_EXTRA_SYMBOLS', " ".join(extra_symbols))
> +    d.setVar('KERNEL_CC', d.getVar('KERNEL_CC').rstrip())
> +    d.setVar('KERNEL_LD', d.getVar('KERNEL_LD').rstrip())
> +    d.setVar('KERNEL_AR', d.getVar('KERNEL_AR').rstrip())
>  }
>

no one else uses KERNEL_* vars, so if modules cant use it then we have
an issue, perhaps just delete it in base class where KERNEL_* vars are set

Re: [PATCH] module.bbclass: swap AR and LD order

[Jason Wessel]

On 1/24/20 3:19 PM, Khem Raj wrote:
> On Fri, Jan 24, 2020 at 1:05 PM Jason Wessel <jason.wessel@windriver.com> wrote:
>>
>> On 1/24/20 1:13 PM, Khem Raj wrote:
>>> On 1/24/20 10:19 AM, Christopher Larson wrote:
>>>> What makefile change caused this? That behavior doesn't make much sense
>>>> given how make processes its command-line arguments.
>>>>
>>>
>>> I agree with you here, it could be a rare check where one want to define
>>> what collect progam should be used ( ld or ar )
>>>
>>> besides, recently we moved AR to be gcc-ar/llvm-ar by default in config
>>> metadata, which is not going to work out of box for compiling kernel and
>>> modules, so overriding it with KERNEL_AR in
>>> module_do_compile/install/configure tasks is good change.
>>
>> I am not actually sure which part of kbuild is parsing the LD
>> arguments.  My best guess is that it is actually all the nested
>> gnumake filter calls causing the issue.
>>
>> With some further experimentation I determined that a space at the end
>> of the LD= is what causes it to think the next argument is part of the
>> prior one.  The poky distro is defining KERNEL_LD as:
>>
>> % bitbake vboxguestdrivers -e |grep ^KERNEL_LD
>> KERNEL_LD="x86_64-poky-linux-ld.bfd "
>>
>> The kbuild really doesn't like that space at the end.  You can trash out
>> a host native kernel compile module build by adding the arguments:
>>     LD="ld " JUNK_VAR="junk"
>>
>> Attempting to debug/fix kbuild doesn't seem like the right answer.
>>
>> The next version of this patch looks like what is below.  Would that be ok?
>>
>>
>> diff --git a/meta/classes/module.bbclass b/meta/classes/module.bbclass
>> index c0dfa35061..7b0a85a06d 100644
>> --- a/meta/classes/module.bbclass
>> +++ b/meta/classes/module.bbclass
>> @@ -12,6 +12,9 @@ python __anonymous () {
>>          if dep.startswith("kernel-module-"):
>>              extra_symbols.append("${STAGING_INCDIR}/" + dep + "/Module.symvers")
>>      d.setVar('KBUILD_EXTRA_SYMBOLS', " ".join(extra_symbols))
>> +    d.setVar('KERNEL_CC', d.getVar('KERNEL_CC').rstrip())
>> +    d.setVar('KERNEL_LD', d.getVar('KERNEL_LD').rstrip())
>> +    d.setVar('KERNEL_AR', d.getVar('KERNEL_AR').rstrip())
>>  }
>>
> 
> no one else uses KERNEL_* vars, so if modules cant use it then we have
> an issue, perhaps just delete it in base class where KERNEL_* vars are set
> 


I am not sure I follow what you want to see changed.  Do you want the to try
and fix up the problem where the variables are initially set?  Example:

classes/kernel-arch.bbclass:KERNEL_LD = "${CCACHE}${HOST_PREFIX}ld.bfd ${HOST_LD_KERNEL_ARCH}"

I see there are some BSPs that append to KERNEL_LD, so this problem could happen
at a later point. 

The other possibility is adding an echo to force the trimming without using the python.

@@ -41,8 +38,8 @@ module_do_compile() {
        unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS
        oe_runmake KERNEL_PATH=${STAGING_KERNEL_DIR}   \
                   KERNEL_VERSION=${KERNEL_VERSION}    \
-                  CC="${KERNEL_CC}" LD="${KERNEL_LD}" \
-                  AR="${KERNEL_AR}" \
+                  CC="$(echo ${KERNEL_CC})" LD="$(echo ${KERNEL_LD})" \
+                  AR="$(echo ${KERNEL_AR})" \
                   O=${STAGING_KERNEL_BUILDDIR} \
                   KBUILD_EXTRA_SYMBOLS="${KBUILD_EXTRA_SYMBOLS}" \
                   ${MAKE_TARGETS}
@@ -52,7 +49,7 @@ module_do_install() {
        unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS
        oe_runmake DEPMOD=echo MODLIB="${D}${nonarch_base_libdir}/modules/${KERNEL_VERSION}" \
                   INSTALL_FW_PATH="${D}${nonarch_base_libdir}/firmware" \
-                  CC="${KERNEL_CC}" LD="${KERNEL_LD}" \
+                  CC="$(echo ${KERNEL_CC})" LD="$(echo ${KERNEL_LD})" \
                   O=${STAGING_KERNEL_BUILDDIR} \
                   ${MODULES_INSTALL_TARGET}