[Buildroot] [PATCH] linux custom tarball: automatically detect a change in tarball configuration

[Buildroot] [PATCH] linux custom tarball: automatically detect a change in tarball configuration

[Thomas De Schampheleire]

When you specify a LINUX_TARBALL, the variable LINUX_VERSION will be set to
the fixed "custom". As a result, LINUX_DIR remains the same even though you
modified the .config file to specify a different tarball.  When re-running
make, buildroot does not see the change because the stamp files still look
up-to-date, and you'll have to force building the new kernel in some way.

This patch overwrites LINUX_VERSION to include the LINUX_TARBALL string, If a
new custom tarball is configured, LINUX_DIR will be different, the old stamp
files will not matter, and buildroot will correctly start the build process for
the new kernel.

Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
---

 linux/linux.mk |  4 ++++
 1 files changed, 4 insertions(+), 0 deletions(-)

diff --git a/linux/linux.mk b/linux/linux.mk
--- a/linux/linux.mk
+++ b/linux/linux.mk
@@ -10,6 +10,10 @@
 LINUX_TARBALL = $(call qstrip,$(BR2_LINUX_KERNEL_CUSTOM_TARBALL_LOCATION))
 LINUX_SITE = $(dir $(LINUX_TARBALL))
 LINUX_SOURCE = $(notdir $(LINUX_TARBALL))
+# Overwrite LINUX_VERSION so that LINUX_DIR becomes unique, in which case a
+# change in the LINUX_TARBALL configuration automatically gets picked up.
+LINUX_VERSION = custom-$(LINUX_TARBALL)
+
 else ifeq ($(BR2_LINUX_KERNEL_CUSTOM_GIT),y)
 LINUX_SITE = $(call qstrip,$(BR2_LINUX_KERNEL_CUSTOM_GIT_REPO_URL))
 LINUX_SITE_METHOD = git

[Buildroot] [PATCH] linux custom tarball: automatically detect a change in tarball configuration

[Thomas Petazzoni]

Le Wed, 21 Sep 2011 10:55:54 +0200,
Thomas De Schampheleire <patrickdepinguin+buildroot@gmail.com> a ?crit :

> When you specify a LINUX_TARBALL, the variable LINUX_VERSION will be set to
> the fixed "custom". As a result, LINUX_DIR remains the same even though you
> modified the .config file to specify a different tarball.  When re-running
> make, buildroot does not see the change because the stamp files still look
> up-to-date, and you'll have to force building the new kernel in some way.
> 
> This patch overwrites LINUX_VERSION to include the LINUX_TARBALL string, If a
> new custom tarball is configured, LINUX_DIR will be different, the old stamp
> files will not matter, and buildroot will correctly start the build process for
> the new kernel.

Sorry, but LINUX_TARBALL can be something like
http://foobar.com/blah/blek/linux-3.11.0.tar.bz2, and I don't think
it's a good idea to create a directory named
linux-custom-http://foobar.com/blah/blek/linux-3.11.0.tar.bz2

You could do something like:

LINUX_VERSION = $(LINUX_SOURCE), in which case the directory would be:

linux-linux-3.11.0.tar.bz2/ in the case highlighted above, which is not
very nice either.

Or:

LINUX_VERSION = $(firstword $(shell echo $(LINUX_TARBALL) | md5sum)

which would create a hash from the LINUX_TARBALL URL, and that hash
would change when the LINUX_TARBALL URL is changed. I don't know if
it's nicer.

I agree that their is a minor inconvenience in having the directory
always named linux-custom/, but I don't have a good solution in mind.

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[Buildroot] [PATCH] linux custom tarball: automatically detect a change in tarball configuration

[Thomas De Schampheleire]

Hi Thomas,

On Thu, Sep 22, 2011 at 9:58 PM, Thomas Petazzoni
<thomas.petazzoni@free-electrons.com> wrote:
> Le Wed, 21 Sep 2011 10:55:54 +0200,
> Thomas De Schampheleire <patrickdepinguin+buildroot@gmail.com> a ?crit :
>
>> When you specify a LINUX_TARBALL, the variable LINUX_VERSION will be set to
>> the fixed "custom". As a result, LINUX_DIR remains the same even though you
>> modified the .config file to specify a different tarball. ?When re-running
>> make, buildroot does not see the change because the stamp files still look
>> up-to-date, and you'll have to force building the new kernel in some way.
>>
>> This patch overwrites LINUX_VERSION to include the LINUX_TARBALL string, If a
>> new custom tarball is configured, LINUX_DIR will be different, the old stamp
>> files will not matter, and buildroot will correctly start the build process for
>> the new kernel.
>
> Sorry, but LINUX_TARBALL can be something like
> http://foobar.com/blah/blek/linux-3.11.0.tar.bz2, and I don't think
> it's a good idea to create a directory named
> linux-custom-http://foobar.com/blah/blek/linux-3.11.0.tar.bz2
>

Ai, I didn't think of that. In my case, I'm relying on the
BR2_PRIMARY_SITE to know where the tarball is at.
The directory that's created in that case would be:
linux-custom-linux-2.XXX-myproject.tar.bz2

> You could do something like:
>
> LINUX_VERSION = $(LINUX_SOURCE), in which case the directory would be:
>
> linux-linux-3.11.0.tar.bz2/ in the case highlighted above, which is not
> very nice either.

I agree that these directory names do not look very good. But they do
uniquely map a custom tarball on a directory, and cause the build
process to detect a change in tarball.
Personally, I don't mind having such a funny-looking directory in output/build.

>
> Or:
>
> LINUX_VERSION = $(firstword $(shell echo $(LINUX_TARBALL) | md5sum)
>
> which would create a hash from the LINUX_TARBALL URL, and that hash
> would change when the LINUX_TARBALL URL is changed. I don't know if
> it's nicer.

This would also do the trick, but has the downside of not being
user-readable. If a user changes from tarball, he will have two linux
directories in output/build, and no direct way of knowing which is the
old and new one (except for looking at the timestamp or calculating
the md5sum himself).

>
> I agree that their is a minor inconvenience in having the directory
> always named linux-custom/, but I don't have a good solution in mind.

The other solution that I implemented in buildroot-2010.11 is to make
the download step depend on the tarball, instead of on a stamp file.
Since linux now uses gentargets, this would require a change in the
whole of gentargets as well.
Essentially, the .stamp_downloaded file would be no longer needed, and
the tarball would serve as stamp file.

Would you prefer such a solution?

Best regards,
Thomas

[Buildroot] [PATCH] linux custom tarball: automatically detect a change in tarball configuration

[Arnout Vandecappelle]

On Friday 23 September 2011 08:16:09, Thomas De Schampheleire wrote:
> > I agree that their is a minor inconvenience in having the directory
> > always named linux-custom/, but I don't have a good solution in mind.
> 
> The other solution that I implemented in buildroot-2010.11 is to make
> the download step depend on the tarball, instead of on a stamp file.
> Since linux now uses gentargets, this would require a change in the
> whole of gentargets as well.
> Essentially, the .stamp_downloaded file would be no longer needed, and
> the tarball would serve as stamp file.
> 
> Would you prefer such a solution?

 I certainly would.

 I never really understood why the dependencies are between the phony targets 
instead of the stamp files.  Otherwise, I could just touch a stamp file to 
force all subsequent build steps to run.  However, I guess the buildroot 
masters must have their reasons :-)

 Regards,
 Arnout

-- 
Arnout Vandecappelle                               arnout at mind be
Senior Embedded Software Architect                 +32-16-286540
Essensium/Mind                                     http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium                BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  31BB CF53 8660 6F88 345D  54CC A836 5879 20D7 CF43
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20110927/54579cfd/attachment-0001.html>

[Buildroot] [PATCH] linux custom tarball: automatically detect a change in tarball configuration

[Daniel Nyström]

On Thu, Sep 22, 2011 at 9:58 PM, Thomas Petazzoni
<thomas.petazzoni@free-electrons.com> wrote:
> Le Wed, 21 Sep 2011 10:55:54 +0200,
> Thomas De Schampheleire <patrickdepinguin+buildroot@gmail.com> a ?crit :
>
>> When you specify a LINUX_TARBALL, the variable LINUX_VERSION will be set to
>> the fixed "custom". As a result, LINUX_DIR remains the same even though you
>> modified the .config file to specify a different tarball. ?When re-running
>> make, buildroot does not see the change because the stamp files still look
>> up-to-date, and you'll have to force building the new kernel in some way.
>>
>> This patch overwrites LINUX_VERSION to include the LINUX_TARBALL string, If a
>> new custom tarball is configured, LINUX_DIR will be different, the old stamp
>> files will not matter, and buildroot will correctly start the build process for
>> the new kernel.
>
> Sorry, but LINUX_TARBALL can be something like
> http://foobar.com/blah/blek/linux-3.11.0.tar.bz2, and I don't think
> it's a good idea to create a directory named
> linux-custom-http://foobar.com/blah/blek/linux-3.11.0.tar.bz2

$ echo "http://foobar.com/blah/blek/linux-3.11.0.tar.bz2" | sed -e
's|^[a-z]*://||;s|[/]|_|g'
foobar.com_blah_blek_linux-3.11.0.tar.bz2

How would "output/build/linux-foobar.com_blah_blek_linux-3.11.0.tar.bz2/" work?