[PATCH 0/1] Fix external module compilations

[PATCH 0/1] Fix external module compilations

[Anders Darander]

When compiling an external module, the compilation errors out, when
scripts/basic/fixdep and later on scripts/mod/modpost is being requested. The
removal of 'make -C $kerneldir _mrproper_scripts' fixes these problems.

The following changes since commit 69eeb3d2276e5b10d084b47d308ecfc8daf8b467:

  qt4: ensure target packages don't include host binaries (2011-07-05 11:28:59 +0100)

are available in the git repository at:
  git://github.com/darander/oe-core fixdep
  https://github.com/darander/oe-core/tree/fixdep

Anders Darander (1):
  kernel.bbclass: make external module compile

 meta/classes/kernel.bbclass |    1 -
 1 files changed, 0 insertions(+), 1 deletions(-)

-- 
1.7.4.1

[PATCH 1/1] kernel.bbclass: make external module compile

[Anders Darander]

When compiling external modules, scripts/basic/fixdep and scripts/mod/modpost, is wanted.
Do not remove too much from the staged kernel sources.

Signed-off-by: Anders Darander <anders@chargestorm.se>
---
 meta/classes/kernel.bbclass |    1 -
 1 files changed, 0 insertions(+), 1 deletions(-)

diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
index 943252a..26ee416 100644
--- a/meta/classes/kernel.bbclass
+++ b/meta/classes/kernel.bbclass
@@ -149,7 +149,6 @@ kernel_do_install() {
 	# and include files.
 	#
 	oe_runmake -C $kerneldir CC="${KERNEL_CC}" LD="${KERNEL_LD}" clean
-	make -C $kerneldir _mrproper_scripts
 	find $kerneldir -path $kerneldir/scripts -prune -o -name "*.[csS]" -exec rm '{}' \;
 	find $kerneldir/Documentation -name "*.txt" -exec rm '{}' \;
 
-- 
1.7.4.1

Re: [PATCH 1/1] kernel.bbclass: make external module compile

[Phil Blundell]

On Tue, 2011-07-05 at 14:01 +0200, Anders Darander wrote:
> diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
> index 943252a..26ee416 100644
> --- a/meta/classes/kernel.bbclass
> +++ b/meta/classes/kernel.bbclass
> @@ -149,7 +149,6 @@ kernel_do_install() {

        #
        # We don't want to leave host-arch binaries in /sysroots, so
        # we clean the scripts dir while leaving the generated config

>  	# and include files.
>  	#
>  	oe_runmake -C $kerneldir CC="${KERNEL_CC}" LD="${KERNEL_LD}" clean
> -	make -C $kerneldir _mrproper_scripts
>  	find $kerneldir -path $kerneldir/scripts -prune -o -name "*.[csS]" -exec rm '{}' \;
>  	find $kerneldir/Documentation -name "*.txt" -exec rm '{}' \;

Did you verify that this doesn't introduce any new QA warnings during
packaging?  Presumably that line was originally added for a reason and
it seems a bit surprising that just deleting it without any replacement
is the right thing to do.

Also, if the scripts dir isn't being cleaned anymore, I guess the
preceding comment should be adjusted to match the new reality.

p.

Re: [PATCH 1/1] kernel.bbclass: make external module compile

[Anders Darander]

* Phil Blundell Phil Blundell <pb@pbcl.net> [07/05/11 02:44 PM]:
> On Tue, 2011-07-05 at 14:01 +0200, Anders Darander wrote:
> > diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
> > index 943252a..26ee416 100644
> > --- a/meta/classes/kernel.bbclass
> > +++ b/meta/classes/kernel.bbclass
> > @@ -149,7 +149,6 @@ kernel_do_install() {
> 
>         #
>         # We don't want to leave host-arch binaries in /sysroots, so
>         # we clean the scripts dir while leaving the generated config
> 
> >  	# and include files.
> >  	#
> >  	oe_runmake -C $kerneldir CC="${KERNEL_CC}" LD="${KERNEL_LD}" clean
> > 
> > -	make -C $kerneldir _mrproper_scripts
> > 
> >  	find $kerneldir -path $kerneldir/scripts -prune -o -name "*.[csS]"
> >  	-exec rm '{}' \; find $kerneldir/Documentation -name "*.txt" -exec rm
> >  	'{}' \;
> 
> Did you verify that this doesn't introduce any new QA warnings during
> packaging?  Presumably that line was originally added for a reason and
> it seems a bit surprising that just deleting it without any replacement
> is the right thing to do.

No, I didn't really verify that. Do I need to run with any specific options 
enabled, or should it be enough to just bitbake my modules recipe? (I can't 
test for the moment, as the latest pull from oe-core forces a rebuild of gcc 
etc).
 
> Also, if the scripts dir isn't being cleaned anymore, I guess the
> preceding comment should be adjusted to match the new reality.

That's true.

I'll wait to see if someone else has any comments, or if I find some QA 
warnings before I produce a version 2.

Regards,
Anders

Re: [PATCH 1/1] kernel.bbclass: make external module compile

[Richard Purdie]

On Tue, 2011-07-05 at 14:54 +0200, Anders Darander wrote:
> * Phil Blundell Phil Blundell <pb@pbcl.net> [07/05/11 02:44 PM]:
> > On Tue, 2011-07-05 at 14:01 +0200, Anders Darander wrote:
> > > diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
> > > index 943252a..26ee416 100644
> > > --- a/meta/classes/kernel.bbclass
> > > +++ b/meta/classes/kernel.bbclass
> > > @@ -149,7 +149,6 @@ kernel_do_install() {
> > 
> >         #
> >         # We don't want to leave host-arch binaries in /sysroots, so
> >         # we clean the scripts dir while leaving the generated config
> > 
> > >  	# and include files.
> > >  	#
> > >  	oe_runmake -C $kerneldir CC="${KERNEL_CC}" LD="${KERNEL_LD}" clean
> > > 
> > > -	make -C $kerneldir _mrproper_scripts
> > > 
> > >  	find $kerneldir -path $kerneldir/scripts -prune -o -name "*.[csS]"
> > >  	-exec rm '{}' \; find $kerneldir/Documentation -name "*.txt" -exec rm
> > >  	'{}' \;
> > 
> > Did you verify that this doesn't introduce any new QA warnings during
> > packaging?  Presumably that line was originally added for a reason and
> > it seems a bit surprising that just deleting it without any replacement
> > is the right thing to do.
> 
> No, I didn't really verify that. Do I need to run with any specific options 
> enabled, or should it be enough to just bitbake my modules recipe? (I can't 
> test for the moment, as the latest pull from oe-core forces a rebuild of gcc 
> etc).
>  
> > Also, if the scripts dir isn't being cleaned anymore, I guess the
> > preceding comment should be adjusted to match the new reality.
> 
> That's true.
> 
> I'll wait to see if someone else has any comments, or if I find some QA 
> warnings before I produce a version 2.

I'm cc'ing Darren as this is one of his favourite subjects :/.

Summary is that this works well in some kernel versions and not in
others. We might have to start doing this conditionally based upon
kernel version I guess...

Cheers,

Richard

Re: [PATCH 1/1] kernel.bbclass: make external module compile

[Anders Darander]

On 5 jul 2011, at 15:09, "Richard Purdie" <richard.purdie@linuxfoundation.org> wrote:

> On Tue, 2011-07-05 at 14:54 +0200, Anders Darander wrote:
>> * Phil Blundell Phil Blundell <pb@pbcl.net> [07/05/11 02:44 PM]:
>>> On Tue, 2011-07-05 at 14:01 +0200, Anders Darander wrote:
>>>> diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
>>>> index 943252a..26ee416 100644
>>>> --- a/meta/classes/kernel.bbclass
>>>> +++ b/meta/classes/kernel.bbclass
>>>> @@ -149,7 +149,6 @@ kernel_do_install() {
>>> 
>>>        #
>>>        # We don't want to leave host-arch binaries in /sysroots, so
>>>        # we clean the scripts dir while leaving the generated config
>>> 
>>>>    # and include files.
>>>>    #
>>>>    oe_runmake -C $kerneldir CC="${KERNEL_CC}" LD="${KERNEL_LD}" clean
>>>> 
>>>> -    make -C $kerneldir _mrproper_scripts
>>>> 
>>>>    find $kerneldir -path $kerneldir/scripts -prune -o -name "*.[csS]"
>>>>    -exec rm '{}' \; find $kerneldir/Documentation -name "*.txt" -exec rm
>>>>    '{}' \;
>>> 
>>> Did you verify that this doesn't introduce any new QA warnings during
>>> packaging?  Presumably that line was originally added for a reason and
>>> it seems a bit surprising that just deleting it without any replacement
>>> is the right thing to do.
>> 
>> No, I didn't really verify that. Do I need to run with any specific options 
>> enabled, or should it be enough to just bitbake my modules recipe? (I can't 
>> test for the moment, as the latest pull from oe-core forces a rebuild of gcc 
>> etc).
>> 
>>> Also, if the scripts dir isn't being cleaned anymore, I guess the
>>> preceding comment should be adjusted to match the new reality.
>> 
>> That's true.
>> 
>> I'll wait to see if someone else has any comments, or if I find some QA 
>> warnings before I produce a version 2.
> 
> I'm cc'ing Darren as this is one of his favourite subjects :/.
> 
> Summary is that this works well in some kernel versions and not in
> others. We might have to start doing this conditionally based upon
> kernel version I guess...

Ah, well then it is worse than I thought. ( On the other hand, that would explain why no one has noticed the problem, until I tried using the wrong kernel version).

Do we have any idea at what point the kernel breaks? I guess that might be a question for Darren.

Cheers,
Anders

Re: [PATCH 1/1] kernel.bbclass: make external module compile

[Darren Hart]

On 07/05/2011 08:04 AM, Anders Darander wrote:
> 
> 
> On 5 jul 2011, at 15:09, "Richard Purdie"
> <richard.purdie@linuxfoundation.org> wrote:
> 
>> On Tue, 2011-07-05 at 14:54 +0200, Anders Darander wrote:
>>> * Phil Blundell Phil Blundell <pb@pbcl.net> [07/05/11 02:44 PM]:
>>>> On Tue, 2011-07-05 at 14:01 +0200, Anders Darander wrote:
>>>>> diff --git a/meta/classes/kernel.bbclass
>>>>> b/meta/classes/kernel.bbclass index 943252a..26ee416 100644 
>>>>> --- a/meta/classes/kernel.bbclass +++
>>>>> b/meta/classes/kernel.bbclass @@ -149,7 +149,6 @@
>>>>> kernel_do_install() {
>>>> 
>>>> # # We don't want to leave host-arch binaries in /sysroots, so 
>>>> # we clean the scripts dir while leaving the generated config
>>>> 
>>>>> # and include files. # oe_runmake -C $kerneldir
>>>>> CC="${KERNEL_CC}" LD="${KERNEL_LD}" clean
>>>>> 
>>>>> -    make -C $kerneldir _mrproper_scripts
>>>>> 
>>>>> find $kerneldir -path $kerneldir/scripts -prune -o -name
>>>>> "*.[csS]" -exec rm '{}' \; find $kerneldir/Documentation
>>>>> -name "*.txt" -exec rm '{}' \;
>>>> 
>>>> Did you verify that this doesn't introduce any new QA warnings
>>>> during packaging?  Presumably that line was originally added
>>>> for a reason and it seems a bit surprising that just deleting
>>>> it without any replacement is the right thing to do.
>>> 
>>> No, I didn't really verify that. Do I need to run with any
>>> specific options enabled, or should it be enough to just bitbake
>>> my modules recipe? (I can't test for the moment, as the latest
>>> pull from oe-core forces a rebuild of gcc etc).
>>> 
>>>> Also, if the scripts dir isn't being cleaned anymore, I guess
>>>> the preceding comment should be adjusted to match the new
>>>> reality.
>>> 
>>> That's true.
>>> 
>>> I'll wait to see if someone else has any comments, or if I find
>>> some QA warnings before I produce a version 2.
>> 
>> I'm cc'ing Darren as this is one of his favourite subjects :/.
>> 
>> Summary is that this works well in some kernel versions and not in 
>> others. We might have to start doing this conditionally based upon 
>> kernel version I guess...
> 
> Ah, well then it is worse than I thought. ( On the other hand, that
> would explain why no one has noticed the problem, until I tried using
> the wrong kernel version).
> 
> Do we have any idea at what point the kernel breaks? I guess that
> might be a question for Darren.

Hi Anders,

Please see the following commit log:

commit 3b49416fc7a7ee9bfe722f2e6089aa18df41dc58
Author: Darren Hart <dvhart@linux.intel.com>
Date:   Tue Mar 8 17:09:10 2011 -0800

    kernel/bbclass: rework kernel and module classes to allow for building out-of-tree modul
  

In particular, the following:

    Care is also taken to clean the hostprogs in scripts, and the modules are
    responsible for building them as needed. Although it is unclear to me if this is
    really necessary, especially considering that modules put these bits back as
    soon as they compile. If we are not generating an sstate package, I suspect we
    can ignore these.
    
The scripts are recreated during the build of module.bbclass derived recipes.
Are you trying to build modules independently of this method? Richard expressed
concerns about not including host specific binaries in the sstate, which was
part of the reason this approach was taken.

--
Darren

> 
> Cheers, Anders _______________________________________________ 
> Openembedded-core mailing list 
> Openembedded-core@lists.openembedded.org 
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core

--
> 
Darren Hart
Intel Open Source Technology Center
Yocto Project - Linux Kernel

Re: [PATCH 1/1] kernel.bbclass: make external module compile

[Anders Darander]

Hi Darren,

On 6 jul 2011, at 18:37, "Darren Hart" <dvhart@linux.intel.com> wrote:
> Please see the following commit log:
> 
> commit 3b49416fc7a7ee9bfe722f2e6089aa18df41dc58
> Author: Darren Hart <dvhart@linux.intel.com>
> Date:   Tue Mar 8 17:09:10 2011 -0800
> 
>    kernel/bbclass: rework kernel and module classes to allow for building out-of-tree modul
> 
> 
> In particular, the following:
> 
>    Care is also taken to clean the hostprogs in scripts, and the modules are
>    responsible for building them as needed. Although it is unclear to me if this is
>    really necessary, especially considering that modules put these bits back as
>    soon as they compile. If we are not generating an sstate package, I suspect we
>    can ignore these.
> 
> The scripts are recreated during the build of module.bbclass derived recipes.
> Are you trying to build modules independently of this method? Richard expressed
> concerns about not including host specific binaries in the sstate, which was
> part of the reason this approach was taken.

Thanks for pointing out this, especially the first sentence in this paragraph. No matter how many times I've looked at both the recipes in question, as well as the meta/classes directory, I haven't noticed that the old, inherited recipes were not using module.bbclass, but rather inherited module-base.bbclass directly. I'll assume that once I change that, and fix the recipes properly all my issues will go away.

Once more, thanks for pointing me to the real problem.

Cheers,
Anders

Re: [PATCH 1/1] kernel.bbclass: make external module compile

[Darren Hart]

On 07/06/2011 10:31 AM, Anders Darander wrote:
> Hi Darren,
> 
> On 6 jul 2011, at 18:37, "Darren Hart" <dvhart@linux.intel.com>
> wrote:
>> Please see the following commit log:
>> 
>> commit 3b49416fc7a7ee9bfe722f2e6089aa18df41dc58 Author: Darren Hart
>> <dvhart@linux.intel.com> Date:   Tue Mar 8 17:09:10 2011 -0800
>> 
>> kernel/bbclass: rework kernel and module classes to allow for
>> building out-of-tree modul
>> 
>> 
>> In particular, the following:
>> 
>> Care is also taken to clean the hostprogs in scripts, and the
>> modules are responsible for building them as needed. Although it is
>> unclear to me if this is really necessary, especially considering
>> that modules put these bits back as soon as they compile. If we are
>> not generating an sstate package, I suspect we can ignore these.
>> 
>> The scripts are recreated during the build of module.bbclass
>> derived recipes. Are you trying to build modules independently of
>> this method? Richard expressed concerns about not including host
>> specific binaries in the sstate, which was part of the reason this
>> approach was taken.
> 
> Thanks for pointing out this, especially the first sentence in this
> paragraph. No matter how many times I've looked at both the recipes
> in question, as well as the meta/classes directory, I haven't noticed
> that the old, inherited recipes were not using module.bbclass, but
> rather inherited module-base.bbclass directly. I'll assume that once
> I change that, and fix the recipes properly all my issues will go
> away.
> 
> Once more, thanks for pointing me to the real problem.

Excellent, please keep us posted!

-- 
Darren Hart
Intel Open Source Technology Center
Yocto Project - Linux Kernel

Re: [PATCH 1/1] kernel.bbclass: make external module compile

[Anders Darander]

On 6 jul 2011, at 20:32, "Darren Hart" <dvhart@linux.intel.com> wrote:
> 
> On 07/06/2011 10:31 AM, Anders Darander wrote:
>> Hi Darren,
>> 
>> On 6 jul 2011, at 18:37, "Darren Hart" <dvhart@linux.intel.com>
>> wrote:
>>> The scripts are recreated during the build of module.bbclass
>>> derived recipes. Are you trying to build modules independently of
>>> this method? Richard expressed concerns about not including host
>>> specific binaries in the sstate, which was part of the reason this
>>> approach was taken.
>> 
>> Thanks for pointing out this, especially the first sentence in this
>> paragraph. No matter how many times I've looked at both the recipes
>> in question, as well as the meta/classes directory, I haven't noticed
>> that the old, inherited recipes were not using module.bbclass, but
>> rather inherited module-base.bbclass directly. I'll assume that once
>> I change that, and fix the recipes properly all my issues will go
>> away.
>> 
>> Once more, thanks for pointing me to the real problem.
> 
> Excellent, please keep us posted!

Fixing the module recipes as hinted above, solved my problems. Thus, I've back ported the fixes to our current build system, based on oe-dev.

Thanks again!

Cheers,
Anders