[Buildroot] genconfig/unconditonally include external.mk

[Buildroot] genconfig/unconditonally include external.mk

[Michael Walle]

Hi,

there was a discussion (and rejected patches) [1] about adding support 
for buildroot config fragments. The conclusion was that this should not 
be part of buildroot and should stay external. But an integration would 
need unconditonal inclusion of the external.mk. The last mail about this 
subject was in July [2]. Are there any news on this topic?

[1] http://patchwork.ozlabs.org/patch/641840/
[2] http://lists.busybox.net/pipermail/buildroot/2016-July/167059.html

-michael

[Buildroot] genconfig/unconditonally include external.mk

[Arnout Vandecappelle]

On 07-11-16 10:49, Michael Walle wrote:
> Hi,
> 
> there was a discussion (and rejected patches) [1] about adding support for
> buildroot config fragments. The conclusion was that this should not be part of
> buildroot and should stay external.

 With "stay external", the idea was not that it would be handled in external.mk,
but rather that you have a script that merges the defconfigs to a temporary file
and the calls 'make defconfig BR2_DEFCONFIG=...' on it.


> But an integration would need unconditonal
> inclusion of the external.mk. The last mail about this subject was in July [2].
> Are there any news on this topic?

 Nobody ever posted a patch :-)

 But I can already tell you that you won't be able to just move the include
outside of the conditional: it can go up over the definition of the package
infras, and it can't go down over the target-finalize: $(PACKAGES) rule. So it
has to stay where it is, and the conditional has to be split up.

 It should be analyzed then what the effect is of including package .mk files
when the package infra is not defined.


 Regards,
 Arnout

> 
> [1] http://patchwork.ozlabs.org/patch/641840/
> [2] http://lists.busybox.net/pipermail/buildroot/2016-July/167059.html
> 
> -michael
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot
> 

-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF

[Buildroot] genconfig/unconditonally include external.mk

[Michael Walle]

Am 2016-11-08 02:22, schrieb Arnout Vandecappelle:
> On 07-11-16 10:49, Michael Walle wrote:
>> Hi,
>> 
>> there was a discussion (and rejected patches) [1] about adding support 
>> for
>> buildroot config fragments. The conclusion was that this should not be 
>> part of
>> buildroot and should stay external.
> 
>  With "stay external", the idea was not that it would be handled in 
> external.mk,
> but rather that you have a script that merges the defconfigs to a 
> temporary file
> and the calls 'make defconfig BR2_DEFCONFIG=...' on it.

mhh, ok, but i guess having additional make targets would be more 
appealing. Esp. if you need some variables defined by buildroot.

>> But an integration would need unconditonal
>> inclusion of the external.mk. The last mail about this subject was in 
>> July [2].
>> Are there any news on this topic?
> 
>  Nobody ever posted a patch :-)
> 
>  But I can already tell you that you won't be able to just move the 
> include
> outside of the conditional: it can go up over the definition of the 
> package
> infras, and it can't go down over the target-finalize: $(PACKAGES) 
> rule. So it
> has to stay where it is, and the conditional has to be split up.
> 
>  It should be analyzed then what the effect is of including package .mk 
> files
> when the package infra is not defined.

mhh I see.. yann wrote that the correct solution would be to include 
external.mk unconditionally. I'm not so sure about this anymore after 
your comment above. I think we should treat the internal and external 
package makefiles the same. That is, either include both if 
BR2_HAVE_DOT_CONFIG is not set or include none of them. But I guess 
there is are reasons not include the package files if 
BR2_HAVE_DOT_CONFIG is not set. So why should be external package .mk 
files be special..

You could include external.mk unconditonally, but put the package .mk 
files in a BR2_HAVE_DOT_CONFIG block, which is not backwards compatible 
and doesn't really fit simplicity. So I'm not sure. I know you were 
already down that route, but a second external.mk which is included 
unconditonally?

-michael

[Buildroot] genconfig/unconditonally include external.mk

[Arnout Vandecappelle]

On 08-11-16 11:59, Michael Walle wrote:
> Am 2016-11-08 02:22, schrieb Arnout Vandecappelle:
>> On 07-11-16 10:49, Michael Walle wrote:
>>> Hi,
>>>
>>> there was a discussion (and rejected patches) [1] about adding support for
>>> buildroot config fragments. The conclusion was that this should not be part of
>>> buildroot and should stay external.
>>
>>  With "stay external", the idea was not that it would be handled in external.mk,
>> but rather that you have a script that merges the defconfigs to a temporary file
>> and the calls 'make defconfig BR2_DEFCONFIG=...' on it.
> 
> mhh, ok, but i guess having additional make targets would be more appealing.
> Esp. if you need some variables defined by buildroot.

 What variables defined by Buildroot could you possibly need? Since you're
making a defconfig, you have no .config available. The only variable that you
may want to use is $(O), but that one you anyway have to pass on the command
line. So pass it as an argument to your merge script. Something like

./genconfig <config-spec> [<path-to-output>]

 It would actually be great if we would have a script like that in Buildroot
that can be used generically.


>>> But an integration would need unconditonal
>>> inclusion of the external.mk. The last mail about this subject was in July [2].
>>> Are there any news on this topic?
>>
>>  Nobody ever posted a patch :-)
>>
>>  But I can already tell you that you won't be able to just move the include
>> outside of the conditional: it can go up over the definition of the package
>> infras, and it can't go down over the target-finalize: $(PACKAGES) rule. So it
>> has to stay where it is, and the conditional has to be split up.
>>
>>  It should be analyzed then what the effect is of including package .mk files
>> when the package infra is not defined.
> 
> mhh I see.. yann wrote that the correct solution would be to include external.mk
> unconditionally. 

 That's not what he wrote. Samuel suggested that it could be done through
external.mk, Patrick said that it was not, and then Yann explained in more
detail why not, and what would need to be changed in order to use external.mk
during configuration. He nowhere said that that would be a good solution. OK, he
wrote "The correct solution would be ..." but (I think) he meant the correct
solution for including external.mk for config, not the correct solution for the
genconfig problem.


> I'm not so sure about this anymore after your comment above. I
> think we should treat the internal and external package makefiles the same. That
> is, either include both if BR2_HAVE_DOT_CONFIG is not set or include none of
> them. But I guess there is are reasons not include the package files if
> BR2_HAVE_DOT_CONFIG is not set. So why should be external package .mk files be
> special..

 That is a very good remark.


> You could include external.mk unconditonally, but put the package .mk files in a
> BR2_HAVE_DOT_CONFIG block, which is not backwards compatible and doesn't really
> fit simplicity. So I'm not sure. I know you were already down that route, but a
> second external.mk which is included unconditonally?

 The problem is that we don't see a sensible use case for it. When we make
changes we want to make sure that we don't break any use cases, so we really
need to know what the use cases are. And use cases that can be solved
differently don't count :-)


 Regards,
 Arnout

-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF

[Buildroot] genconfig/unconditonally include external.mk

[Michael Walle]

Am 2016-11-08 13:02, schrieb Arnout Vandecappelle:
> On 08-11-16 11:59, Michael Walle wrote:
>> Am 2016-11-08 02:22, schrieb Arnout Vandecappelle:
>> mhh, ok, but i guess having additional make targets would be more 
>> appealing.
>> Esp. if you need some variables defined by buildroot.
> 
>  What variables defined by Buildroot could you possibly need? Since 
> you're
> making a defconfig, you have no .config available. The only variable 
> that you
> may want to use is $(O), but that one you anyway have to pass on the 
> command
> line.

Ahh, I thought of the support/kconfig/merge_config.sh for which I would 
need TOPDIR. I did just a quick test and it turns out, that I cannot use 
it as is, because it does an alldefconfig which isn't supported by 
buildroot (yet? dunno).


> So pass it as an argument to your merge script. Something like
> 
> ./genconfig <config-spec> [<path-to-output>]
> 
>  It would actually be great if we would have a script like that in 
> Buildroot
> that can be used generically.


Lets say we have a generic genconfig inside buildroot. Would it be only 
for external users or would you use it to generate similar defconfig 
files inside buildroot? And in the latter case, how would you do it?


>> mhh I see.. yann wrote that the correct solution would be to include 
>> external.mk
>> unconditionally.
> 
>  That's not what he wrote. Samuel suggested that it could be done 
> through
> external.mk, Patrick said that it was not, and then Yann explained in 
> more
> detail why not, and what would need to be changed in order to use 
> external.mk
> during configuration. He nowhere said that that would be a good 
> solution. OK, he
> wrote "The correct solution would be ..." but (I think) he meant the 
> correct
> solution for including external.mk for config, not the correct solution 
> for the
> genconfig problem.
> 
> 
>> I'm not so sure about this anymore after your comment above. I
>> think we should treat the internal and external package makefiles the 
>> same. That
>> is, either include both if BR2_HAVE_DOT_CONFIG is not set or include 
>> none of
>> them. But I guess there is are reasons not include the package files 
>> if
>> BR2_HAVE_DOT_CONFIG is not set. So why should be external package .mk 
>> files be
>> special..
> 
>  That is a very good remark.
> 
> 
>> You could include external.mk unconditonally, but put the package .mk 
>> files in a
>> BR2_HAVE_DOT_CONFIG block, which is not backwards compatible and 
>> doesn't really
>> fit simplicity. So I'm not sure. I know you were already down that 
>> route, but a
>> second external.mk which is included unconditonally?
> 
>  The problem is that we don't see a sensible use case for it. When we 
> make
> changes we want to make sure that we don't break any use cases, so we 
> really
> need to know what the use cases are. And use cases that can be solved
> differently don't count :-)

Yeah and its really ok, because otherwise buildroot will be more and 
more complex, and I don't want it to become another yocto :o) 
Nonetheless, the genconfig wasn't the only use case for this, but also 
an external help command, which also only makes sense, if you have own 
makefile targets, mhh.
And I really liked the idea of having a target "my_defconfig" where 
my_defconfig doesn't have to be a file but some script which do the 
defconfig. IMHO, from a users perspective it makes sense to have the 
same kind of target (*_defconfig).

-michael

[Buildroot] genconfig/unconditonally include external.mk

[Arnout Vandecappelle]

On 08-11-16 13:38, Michael Walle wrote:
> Am 2016-11-08 13:02, schrieb Arnout Vandecappelle:
>> On 08-11-16 11:59, Michael Walle wrote:
>>> Am 2016-11-08 02:22, schrieb Arnout Vandecappelle:
>>> mhh, ok, but i guess having additional make targets would be more appealing.
>>> Esp. if you need some variables defined by buildroot.
>>
>>  What variables defined by Buildroot could you possibly need? Since you're
>> making a defconfig, you have no .config available. The only variable that you
>> may want to use is $(O), but that one you anyway have to pass on the command
>> line.
> 
> Ahh, I thought of the support/kconfig/merge_config.sh for which I would need
> TOPDIR. I did just a quick test and it turns out, that I cannot use it as is,
> because it does an alldefconfig which isn't supported by buildroot (yet? dunno).

 Shouldn't that be olddefconfig (which is supported by buildroot)?

> 
> 
>> So pass it as an argument to your merge script. Something like
>>
>> ./genconfig <config-spec> [<path-to-output>]
>>
>>  It would actually be great if we would have a script like that in Buildroot
>> that can be used generically.
> 
> 
> Lets say we have a generic genconfig inside buildroot. Would it be only for
> external users or would you use it to generate similar defconfig files inside
> buildroot? And in the latter case, how would you do it?

 Both, I think. To use it to generate defconfig files inside buildroot, we'd add
something in the documentation explaining how you can create an extended board
config by calling the script to combine the base config with an extension config.


[snip]
>>  The problem is that we don't see a sensible use case for it. When we make
>> changes we want to make sure that we don't break any use cases, so we really
>> need to know what the use cases are. And use cases that can be solved
>> differently don't count :-)
> 
> Yeah and its really ok, because otherwise buildroot will be more and more
> complex, and I don't want it to become another yocto :o) 

 We started using pseudo, and firing up make takes almost as long as firing up
bitbake, so I'm afraid we're not that far off...


> Nonetheless, the
> genconfig wasn't the only use case for this, but also an external help command,
> which also only makes sense, if you have own makefile targets, mhh.
> And I really liked the idea of having a target "my_defconfig" where my_defconfig
> doesn't have to be a file but some script which do the defconfig. IMHO, from a
> users perspective it makes sense to have the same kind of target (*_defconfig).

 We have a (IMHO healthy) resistance against such "frivolous features", but if
there is enough push for it they eventually tend to come through. Cfr.
BR2_EXTERNAL, for instance. Keep on posting patches and talking to people!

 Regards,
 Arnout


-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF