* [Buildroot] [PATCH 1 of 5] Support URI schemes in DOWNLOAD function, when not using gentargets
2011-09-21 6:55 [Buildroot] [PATCH 0 of 5] toolchain-external: improve support for custom toolchains Thomas De Schampheleire
@ 2011-09-21 6:55 ` Thomas De Schampheleire
2011-09-21 6:55 ` [Buildroot] [PATCH 2 of 5] toolchain-external: allow downloading a custom toolchain Thomas De Schampheleire
` (3 subsequent siblings)
4 siblings, 0 replies; 21+ messages in thread
From: Thomas De Schampheleire @ 2011-09-21 6:55 UTC (permalink / raw)
To: buildroot
When using the gentargets/autotools/cmake infrastructure, the DOWNLOAD function
uses an appropriate download method based on $(PKG)_SITE_METHOD, which is
autodetected based on the URI if none was explicitly set.
When the DOWNLOAD function is called directly, when not using gentargets, the
SITE_METHOD is not autodetected, and thus the download defaults to wget.
This patch adds URI scheme detection directly to the DOWNLOAD method, in case
no SITE_METHOD was detected yet.
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
---
package/Makefile.package.in | 10 +++++++++-
1 files changed, 9 insertions(+), 1 deletions(-)
diff --git a/package/Makefile.package.in b/package/Makefile.package.in
--- a/package/Makefile.package.in
+++ b/package/Makefile.package.in
@@ -85,6 +85,9 @@
DL_DIR:=$(TOPDIR)/dl
endif
+# URI scheme helper function
+geturischeme=$(firstword $(subst ://, ,$(call qstrip,$(1))))
+
################################################################################
# The DOWNLOAD_{GIT,SVN,BZR} helpers are in charge of getting a
# working copy of the source repository for their corresponding SCM,
@@ -189,7 +192,12 @@
$(call $(DL_MODE)_WGET,$(BR2_PRIMARY_SITE),$(2)) && exit ; \
fi ; \
if test -n "$(1)" ; then \
- case "$($(PKG)_SITE_METHOD)" in \
+ if test -z "$($(PKG)_SITE_METHOD)" ; then \
+ scheme="$(call geturischeme,$(1))" ; \
+ else \
+ scheme="$($(PKG)_SITE_METHOD)" ; \
+ fi ; \
+ case "$$scheme" in \
git) $($(DL_MODE)_GIT) && exit ;; \
svn) $($(DL_MODE)_SVN) && exit ;; \
bzr) $($(DL_MODE)_BZR) && exit ;; \
^ permalink raw reply [flat|nested] 21+ messages in thread* [Buildroot] [PATCH 2 of 5] toolchain-external: allow downloading a custom toolchain
2011-09-21 6:55 [Buildroot] [PATCH 0 of 5] toolchain-external: improve support for custom toolchains Thomas De Schampheleire
2011-09-21 6:55 ` [Buildroot] [PATCH 1 of 5] Support URI schemes in DOWNLOAD function, when not using gentargets Thomas De Schampheleire
@ 2011-09-21 6:55 ` Thomas De Schampheleire
2011-09-21 7:58 ` Thomas Petazzoni
2011-09-21 6:55 ` [Buildroot] [PATCH 3 of 5] toolchain-external: allow specifying extra external libraries Thomas De Schampheleire
` (2 subsequent siblings)
4 siblings, 1 reply; 21+ messages in thread
From: Thomas De Schampheleire @ 2011-09-21 6:55 UTC (permalink / raw)
To: buildroot
Custom external toolchains currently already have to be extracted by the user.
This patch adds support for downloading a custom toolchain in the form of a
tarball, which will be extracted by buildroot.
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
---
toolchain/toolchain-external/Config.in | 13 ++++++++++++-
toolchain/toolchain-external/ext-tool.mk | 9 +++++++++
2 files changed, 21 insertions(+), 1 deletions(-)
diff --git a/toolchain/toolchain-external/Config.in b/toolchain/toolchain-external/Config.in
--- a/toolchain/toolchain-external/Config.in
+++ b/toolchain/toolchain-external/Config.in
@@ -193,12 +193,23 @@
config BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD
bool "Download toolchain automatically"
default y
- depends on !BR2_TOOLCHAIN_EXTERNAL_CUSTOM
help
When enabled, Buildroot will automatically download and
install the selected external toolchain. When disabled,
Buildroot will use a pre-installed toolchain.
+config BR2_TOOLCHAIN_EXTERNAL_SITE
+ string "Toolchain download site"
+ depends on BR2_TOOLCHAIN_EXTERNAL_CUSTOM && BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD
+ help
+ Base URL at which the custom toolchain can be downloaded.
+
+config BR2_TOOLCHAIN_EXTERNAL_SOURCE
+ string "Toolchain filename"
+ depends on BR2_TOOLCHAIN_EXTERNAL_CUSTOM && BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD
+ help
+ Filename of the toolchain package to download.
+
config BR2_TOOLCHAIN_EXTERNAL_PATH
string "Toolchain path"
default "/path/to/toolchain/usr"
diff --git a/toolchain/toolchain-external/ext-tool.mk b/toolchain/toolchain-external/ext-tool.mk
--- a/toolchain/toolchain-external/ext-tool.mk
+++ b/toolchain/toolchain-external/ext-tool.mk
@@ -160,6 +160,10 @@
TOOLCHAIN_EXTERNAL_DEPENDENCIES = $(STAMP_DIR)/ext-toolchain-checked
endif
+ifeq ($(BR2_TOOLCHAIN_EXTERNAL_CUSTOM),y)
+TOOLCHAIN_EXTERNAL_DEPENDENCIES += $(STAMP_DIR)/ext-toolchain-checked
+endif
+
ifeq ($(BR2_TOOLCHAIN_EXTERNAL_CODESOURCERY_ARM2009Q1),y)
TOOLCHAIN_EXTERNAL_SITE=http://sourcery.mentor.com/sgpp/lite/arm/portal/package4571/public/arm-none-linux-gnueabi/
TOOLCHAIN_EXTERNAL_SOURCE=arm-2009q1-203-arm-none-linux-gnueabi-i686-pc-linux-gnu.tar.bz2
@@ -197,11 +201,16 @@
TOOLCHAIN_EXTERNAL_SOURCE_2 = blackfin-toolchain-uclibc-full-2010R1-RC4.i386.tar.bz2
TOOLCHAIN_EXTERNAL_SOURCE = $(TOOLCHAIN_EXTERNAL_SOURCE_1) $(TOOLCHAIN_EXTERNAL_SOURCE_2)
else
+# Custom toolchain
+TOOLCHAIN_EXTERNAL_SITE=$(call qstrip,$(BR2_TOOLCHAIN_EXTERNAL_SITE))
+TOOLCHAIN_EXTERNAL_SOURCE=$(call qstrip,$(BR2_TOOLCHAIN_EXTERNAL_SOURCE))
# A value must be set (even if unused), otherwise the
# $(DL_DIR)/$(TOOLCHAIN_EXTERNAL_SOURCE) rule would override the main
# $(DL_DIR) rule
+ifeq (,$(TOOLCHAIN_EXTERNAL_SOURCE))
TOOLCHAIN_EXTERNAL_SOURCE=none
endif
+endif
# Special handling for Blackfin toolchain, because of the split in two
# tarballs, and the organization of tarball contents. The tarballs
^ permalink raw reply [flat|nested] 21+ messages in thread* [Buildroot] [PATCH 2 of 5] toolchain-external: allow downloading a custom toolchain
2011-09-21 6:55 ` [Buildroot] [PATCH 2 of 5] toolchain-external: allow downloading a custom toolchain Thomas De Schampheleire
@ 2011-09-21 7:58 ` Thomas Petazzoni
2011-09-21 8:22 ` Thomas De Schampheleire
0 siblings, 1 reply; 21+ messages in thread
From: Thomas Petazzoni @ 2011-09-21 7:58 UTC (permalink / raw)
To: buildroot
Le Wed, 21 Sep 2011 08:55:32 +0200,
Thomas De Schampheleire <patrickdepinguin+buildroot@gmail.com> a ?crit :
> Custom external toolchains currently already have to be extracted by the user.
> This patch adds support for downloading a custom toolchain in the form of a
> tarball, which will be extracted by buildroot.
I like this feature, but I think I'd prefer to clearly separate the
cases of "local custom toolchain" and "remote custom toolchain".
So, in the toolchain choice :
* All the CodeSourcery toolchains
* Local custom toolchain
* Remote custom toolchain
When Local custom toolchain is selected, show :
* Path of the toolchain
* Prefix of the toolchain
When Remote custom toolchain is selected, show :
* URL of the toolchain tarball (please don't split site and source in
the configuration, we don't do that in any other place in Buildroot,
and it's really strange for the user to have to split the URL in two
parts)
* Prefix of the toolchain
What do you think ?
Thomas
--
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com
^ permalink raw reply [flat|nested] 21+ messages in thread
* [Buildroot] [PATCH 2 of 5] toolchain-external: allow downloading a custom toolchain
2011-09-21 7:58 ` Thomas Petazzoni
@ 2011-09-21 8:22 ` Thomas De Schampheleire
2011-09-21 8:23 ` Thomas De Schampheleire
2011-09-21 11:38 ` Thomas Petazzoni
0 siblings, 2 replies; 21+ messages in thread
From: Thomas De Schampheleire @ 2011-09-21 8:22 UTC (permalink / raw)
To: buildroot
On Wed, Sep 21, 2011 at 9:58 AM, Thomas Petazzoni
<thomas.petazzoni@free-electrons.com> wrote:
> Le Wed, 21 Sep 2011 08:55:32 +0200,
> Thomas De Schampheleire <patrickdepinguin+buildroot@gmail.com> a ?crit :
>
>> Custom external toolchains currently already have to be extracted by the user.
>> This patch adds support for downloading a custom toolchain in the form of a
>> tarball, which will be extracted by buildroot.
>
> I like this feature, but I think I'd prefer to clearly separate the
> cases of "local custom toolchain" and "remote custom toolchain".
>
> So, in the toolchain choice :
>
> ?* All the CodeSourcery toolchains
> ?* Local custom toolchain
> ?* Remote custom toolchain
>
> When Local custom toolchain is selected, show :
> ?* Path of the toolchain
> ?* Prefix of the toolchain
>
> When Remote custom toolchain is selected, show :
> ?* URL of the toolchain tarball (please don't split site and source in
> ? the configuration, we don't do that in any other place in Buildroot,
> ? and it's really strange for the user to have to split the URL in two
> ? parts)
> ?* Prefix of the toolchain
>
> What do you think ?
>
For me it's fine to keep the distinction, but be aware that the
local/remote naming may be confusing: if a user receives a toolchain
tarball from e.g. a hardware manufacturer and saves it to his disk, I
would call it 'local'. But that user would typically like to use the
'remote' rules so that he does not need to unpack the toolchain
himself.
Maybe we should call it 'custom toolchain tarball' versus 'custom
unpacked toolchain' or something alike. The tarball can be either
local (file://something) or remote (http, ftp, scp, ...).
Best regards,
Thomas
> Thomas
> --
> Thomas Petazzoni, Free Electrons
> Kernel, drivers, real-time and embedded Linux
> development, consulting, training and support.
> http://free-electrons.com
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot
^ permalink raw reply [flat|nested] 21+ messages in thread
* [Buildroot] [PATCH 2 of 5] toolchain-external: allow downloading a custom toolchain
2011-09-21 8:22 ` Thomas De Schampheleire
@ 2011-09-21 8:23 ` Thomas De Schampheleire
2011-09-21 11:39 ` Thomas Petazzoni
2011-09-21 11:38 ` Thomas Petazzoni
1 sibling, 1 reply; 21+ messages in thread
From: Thomas De Schampheleire @ 2011-09-21 8:23 UTC (permalink / raw)
To: buildroot
On Wed, Sep 21, 2011 at 10:22 AM, Thomas De Schampheleire
<patrickdepinguin+buildroot@gmail.com> wrote:
> On Wed, Sep 21, 2011 at 9:58 AM, Thomas Petazzoni
> <thomas.petazzoni@free-electrons.com> wrote:
>> Le Wed, 21 Sep 2011 08:55:32 +0200,
>> Thomas De Schampheleire <patrickdepinguin+buildroot@gmail.com> a ?crit :
>>
>>> Custom external toolchains currently already have to be extracted by the user.
>>> This patch adds support for downloading a custom toolchain in the form of a
>>> tarball, which will be extracted by buildroot.
>>
>> I like this feature, but I think I'd prefer to clearly separate the
>> cases of "local custom toolchain" and "remote custom toolchain".
>>
>> So, in the toolchain choice :
>>
>> ?* All the CodeSourcery toolchains
>> ?* Local custom toolchain
>> ?* Remote custom toolchain
>>
>> When Local custom toolchain is selected, show :
>> ?* Path of the toolchain
>> ?* Prefix of the toolchain
>>
>> When Remote custom toolchain is selected, show :
>> ?* URL of the toolchain tarball (please don't split site and source in
>> ? the configuration, we don't do that in any other place in Buildroot,
>> ? and it's really strange for the user to have to split the URL in two
>> ? parts)
>> ?* Prefix of the toolchain
>>
>> What do you think ?
>>
>
> For me it's fine to keep the distinction, but be aware that the
> local/remote naming may be confusing: if a user receives a toolchain
> tarball from e.g. a hardware manufacturer and saves it to his disk, I
> would call it 'local'. But that user would typically like to use the
> 'remote' rules so that he does not need to unpack the toolchain
> himself.
>
> Maybe we should call it 'custom toolchain tarball' versus 'custom
> unpacked toolchain' or something alike. The tarball can be either
> local (file://something) or remote (http, ftp, scp, ...).
By the way, the reason that I made the SITE / SOURCE distinction is
that this is how the CodeSourcery toolchains are handled. If we are to
merge SITE and SOURCE into one, I think we should do it for the
CodeSourcery toolchains as well...
>
> Best regards,
> Thomas
>
>> Thomas
>> --
>> Thomas Petazzoni, Free Electrons
>> Kernel, drivers, real-time and embedded Linux
>> development, consulting, training and support.
>> http://free-electrons.com
>> _______________________________________________
>> buildroot mailing list
>> buildroot at busybox.net
>> http://lists.busybox.net/mailman/listinfo/buildroot
>
^ permalink raw reply [flat|nested] 21+ messages in thread
* [Buildroot] [PATCH 2 of 5] toolchain-external: allow downloading a custom toolchain
2011-09-21 8:23 ` Thomas De Schampheleire
@ 2011-09-21 11:39 ` Thomas Petazzoni
0 siblings, 0 replies; 21+ messages in thread
From: Thomas Petazzoni @ 2011-09-21 11:39 UTC (permalink / raw)
To: buildroot
Le Wed, 21 Sep 2011 10:23:39 +0200,
Thomas De Schampheleire <patrickdepinguin+buildroot@gmail.com> a ?crit :
> By the way, the reason that I made the SITE / SOURCE distinction is
> that this is how the CodeSourcery toolchains are handled. If we are to
> merge SITE and SOURCE into one, I think we should do it for the
> CodeSourcery toolchains as well...
For the CodeSourcery toolchain, the SITE/SOURCE distinction is only
inside the .mk file, i.e it is not visible to the user. My comment was
strictly about what we present to the user in terms of configuration
options, and for this, I'd like to see a single "URL" option rather
than separate SITE/SOURCE.
Now, whether we should have separate SITE/SOURCE or a single URL for
the hardcoded CodeSourcery toolchains, I don't really care. Maybe
having a single URL is simpler, and the URL just gets split in
site/source when calling the download helper.
Regards,
Thomas
--
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com
^ permalink raw reply [flat|nested] 21+ messages in thread
* [Buildroot] [PATCH 2 of 5] toolchain-external: allow downloading a custom toolchain
2011-09-21 8:22 ` Thomas De Schampheleire
2011-09-21 8:23 ` Thomas De Schampheleire
@ 2011-09-21 11:38 ` Thomas Petazzoni
1 sibling, 0 replies; 21+ messages in thread
From: Thomas Petazzoni @ 2011-09-21 11:38 UTC (permalink / raw)
To: buildroot
Le Wed, 21 Sep 2011 10:22:16 +0200,
Thomas De Schampheleire <patrickdepinguin+buildroot@gmail.com> a ?crit :
> For me it's fine to keep the distinction, but be aware that the
> local/remote naming may be confusing: if a user receives a toolchain
> tarball from e.g. a hardware manufacturer and saves it to his disk, I
> would call it 'local'. But that user would typically like to use the
> 'remote' rules so that he does not need to unpack the toolchain
> himself.
>
> Maybe we should call it 'custom toolchain tarball' versus 'custom
> unpacked toolchain' or something alike. The tarball can be either
> local (file://something) or remote (http, ftp, scp, ...).
Yes, sounds better.
Regards,
Thomas
--
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com
^ permalink raw reply [flat|nested] 21+ messages in thread
* [Buildroot] [PATCH 3 of 5] toolchain-external: allow specifying extra external libraries
2011-09-21 6:55 [Buildroot] [PATCH 0 of 5] toolchain-external: improve support for custom toolchains Thomas De Schampheleire
2011-09-21 6:55 ` [Buildroot] [PATCH 1 of 5] Support URI schemes in DOWNLOAD function, when not using gentargets Thomas De Schampheleire
2011-09-21 6:55 ` [Buildroot] [PATCH 2 of 5] toolchain-external: allow downloading a custom toolchain Thomas De Schampheleire
@ 2011-09-21 6:55 ` Thomas De Schampheleire
2011-09-21 7:59 ` Thomas Petazzoni
2011-09-21 6:55 ` [Buildroot] [PATCH 4 of 5] tar: create a host version Thomas De Schampheleire
2011-09-21 6:55 ` [Buildroot] [PATCH 5 of 5] toolchain-external: use host-tar instead of tar to unpack toolchains Thomas De Schampheleire
4 siblings, 1 reply; 21+ messages in thread
From: Thomas De Schampheleire @ 2011-09-21 6:55 UTC (permalink / raw)
To: buildroot
Custom toolchains may provide extra libraries that need to be copied to the
target. This patch adds a configuration option for this.
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
---
toolchain/toolchain-external/Config.in | 8 ++++++++
toolchain/toolchain-external/ext-tool.mk | 1 +
2 files changed, 9 insertions(+), 0 deletions(-)
diff --git a/toolchain/toolchain-external/Config.in b/toolchain/toolchain-external/Config.in
--- a/toolchain/toolchain-external/Config.in
+++ b/toolchain/toolchain-external/Config.in
@@ -333,6 +333,14 @@
support. If you don't know, leave the default value,
Buildroot will tell you if it's correct or not.
+config BR2_TOOLCHAIN_EXTRA_EXTERNAL_LIBS
+ string "Extra toolchain libraries to be copied to target"
+ help
+ If your external toolchain provides extra libraries that
+ need to be copied to the target filesystem, enter them
+ here, separated by spaces. They will be copied to the
+ target's /lib directory.
+
endif # BR2_TOOLCHAIN_EXTERNAL_CUSTOM
endif # BR2_TOOLCHAIN_EXTERNAL
diff --git a/toolchain/toolchain-external/ext-tool.mk b/toolchain/toolchain-external/ext-tool.mk
--- a/toolchain/toolchain-external/ext-tool.mk
+++ b/toolchain/toolchain-external/ext-tool.mk
@@ -51,6 +51,7 @@
# of Buildroot is handled identical for the 2 toolchain types.
LIB_EXTERNAL_LIBS=ld*.so libc.so libcrypt.so libdl.so libgcc_s.so libm.so libnsl.so libresolv.so librt.so libutil.so
+LIB_EXTERNAL_LIBS+=$(BR2_TOOLCHAIN_EXTRA_EXTERNAL_LIBS)
ifeq ($(BR2_TOOLCHAIN_EXTERNAL_GLIBC),y)
LIB_EXTERNAL_LIBS+=libnss_files.so libnss_dns.so
endif
^ permalink raw reply [flat|nested] 21+ messages in thread
* [Buildroot] [PATCH 3 of 5] toolchain-external: allow specifying extra external libraries
2011-09-21 6:55 ` [Buildroot] [PATCH 3 of 5] toolchain-external: allow specifying extra external libraries Thomas De Schampheleire
@ 2011-09-21 7:59 ` Thomas Petazzoni
2011-09-21 8:24 ` Thomas De Schampheleire
2011-09-22 19:07 ` Thomas Petazzoni
0 siblings, 2 replies; 21+ messages in thread
From: Thomas Petazzoni @ 2011-09-21 7:59 UTC (permalink / raw)
To: buildroot
Le Wed, 21 Sep 2011 08:55:33 +0200,
Thomas De Schampheleire <patrickdepinguin+buildroot@gmail.com> a ?crit :
> Custom toolchains may provide extra libraries that need to be copied to the
> target. This patch adds a configuration option for this.
Ok, I will merge into one of my branch with some other external
toolchain improvements and submit the whole thing to Peter.
Thomas
--
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com
^ permalink raw reply [flat|nested] 21+ messages in thread
* [Buildroot] [PATCH 3 of 5] toolchain-external: allow specifying extra external libraries
2011-09-21 7:59 ` Thomas Petazzoni
@ 2011-09-21 8:24 ` Thomas De Schampheleire
2011-09-22 19:07 ` Thomas Petazzoni
1 sibling, 0 replies; 21+ messages in thread
From: Thomas De Schampheleire @ 2011-09-21 8:24 UTC (permalink / raw)
To: buildroot
On Wed, Sep 21, 2011 at 9:59 AM, Thomas Petazzoni
<thomas.petazzoni@free-electrons.com> wrote:
> Le Wed, 21 Sep 2011 08:55:33 +0200,
> Thomas De Schampheleire <patrickdepinguin+buildroot@gmail.com> a ?crit :
>
>> Custom toolchains may provide extra libraries that need to be copied to the
>> target. This patch adds a configuration option for this.
>
> Ok, I will merge into one of my branch with some other external
> toolchain improvements and submit the whole thing to Peter.
Thanks, that's much appreciated!
^ permalink raw reply [flat|nested] 21+ messages in thread
* [Buildroot] [PATCH 3 of 5] toolchain-external: allow specifying extra external libraries
2011-09-21 7:59 ` Thomas Petazzoni
2011-09-21 8:24 ` Thomas De Schampheleire
@ 2011-09-22 19:07 ` Thomas Petazzoni
1 sibling, 0 replies; 21+ messages in thread
From: Thomas Petazzoni @ 2011-09-22 19:07 UTC (permalink / raw)
To: buildroot
Le Wed, 21 Sep 2011 09:59:22 +0200,
Thomas Petazzoni <thomas.petazzoni@free-electrons.com> a ?crit :
> Ok, I will merge into one of my branch with some other external
> toolchain improvements and submit the whole thing to Peter.
Merged.
Thomas
--
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com
^ permalink raw reply [flat|nested] 21+ messages in thread
* [Buildroot] [PATCH 4 of 5] tar: create a host version
2011-09-21 6:55 [Buildroot] [PATCH 0 of 5] toolchain-external: improve support for custom toolchains Thomas De Schampheleire
` (2 preceding siblings ...)
2011-09-21 6:55 ` [Buildroot] [PATCH 3 of 5] toolchain-external: allow specifying extra external libraries Thomas De Schampheleire
@ 2011-09-21 6:55 ` Thomas De Schampheleire
2011-09-21 6:55 ` [Buildroot] [PATCH 5 of 5] toolchain-external: use host-tar instead of tar to unpack toolchains Thomas De Schampheleire
4 siblings, 0 replies; 21+ messages in thread
From: Thomas De Schampheleire @ 2011-09-21 6:55 UTC (permalink / raw)
To: buildroot
Some systems, like RedHat/CentOS, are even today shipped with ancient versions
of GNU tar, like 1.15. This can cause trouble, for example when unpacking
archives containing hardlinks with the --strip-components option.
To avoid such problems, let buildroot create a host-tar that can be used when
unpacking.
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
---
package/tar/tar.mk | 3 +++
1 files changed, 3 insertions(+), 0 deletions(-)
diff --git a/package/tar/tar.mk b/package/tar/tar.mk
--- a/package/tar/tar.mk
+++ b/package/tar/tar.mk
@@ -8,3 +8,6 @@
TAR_SITE = $(BR2_GNU_MIRROR)/tar
$(eval $(call AUTOTARGETS,package,tar))
+$(eval $(call AUTOTARGETS,package,tar,host))
+
+HOST_TAR = $(HOST_DIR)/usr/bin/tar
^ permalink raw reply [flat|nested] 21+ messages in thread* [Buildroot] [PATCH 5 of 5] toolchain-external: use host-tar instead of tar to unpack toolchains
2011-09-21 6:55 [Buildroot] [PATCH 0 of 5] toolchain-external: improve support for custom toolchains Thomas De Schampheleire
` (3 preceding siblings ...)
2011-09-21 6:55 ` [Buildroot] [PATCH 4 of 5] tar: create a host version Thomas De Schampheleire
@ 2011-09-21 6:55 ` Thomas De Schampheleire
2011-09-21 8:02 ` Thomas Petazzoni
4 siblings, 1 reply; 21+ messages in thread
From: Thomas De Schampheleire @ 2011-09-21 6:55 UTC (permalink / raw)
To: buildroot
Some toolchains, like the one built with buildroot itself, use hardlinks (for
example to link between the c++ and g++ binary). Unpacking such a toolchain
with the --strip-components options does not work correctly if the system tar
is too old (<1.17). Even recent releases of RedHat/CentOS still ship with
tar 1.15.
To avoid such problems, always use host-tar instead of tar to unpack
toolchains.
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
---
toolchain/toolchain-external/ext-tool.mk | 11 +++++++----
1 files changed, 7 insertions(+), 4 deletions(-)
diff --git a/toolchain/toolchain-external/ext-tool.mk b/toolchain/toolchain-external/ext-tool.mk
--- a/toolchain/toolchain-external/ext-tool.mk
+++ b/toolchain/toolchain-external/ext-tool.mk
@@ -156,7 +156,10 @@
endif
ifeq ($(BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD),y)
-TOOLCHAIN_EXTERNAL_DEPENDENCIES = $(TOOLCHAIN_EXTERNAL_DIR)/.extracted
+ ifeq (,$(wildcard $(HOST_DIR)/usr/bin/tar))
+ TOOLCHAIN_EXTERNAL_DEPENDENCIES = host-tar
+ endif
+ TOOLCHAIN_EXTERNAL_DEPENDENCIES += $(TOOLCHAIN_EXTERNAL_DIR)/.extracted
else
TOOLCHAIN_EXTERNAL_DEPENDENCIES = $(STAMP_DIR)/ext-toolchain-checked
endif
@@ -228,9 +231,9 @@
$(TOOLCHAIN_EXTERNAL_DIR)/.extracted: $(DL_DIR)/$(TOOLCHAIN_EXTERNAL_SOURCE_1) $(DL_DIR)/$(TOOLCHAIN_EXTERNAL_SOURCE_2)
mkdir -p $(@D)
$(INFLATE$(suffix $(TOOLCHAIN_EXTERNAL_SOURCE_1))) $(DL_DIR)/$(TOOLCHAIN_EXTERNAL_SOURCE_1) | \
- $(TAR) $(TAR_STRIP_COMPONENTS)=3 --hard-dereference -C $(@D) $(TAR_OPTIONS) -
+ $(HOST_TAR) $(TAR_STRIP_COMPONENTS)=3 --hard-dereference -C $(@D) $(TAR_OPTIONS) -
$(INFLATE$(suffix $(TOOLCHAIN_EXTERNAL_SOURCE_2))) $(DL_DIR)/$(TOOLCHAIN_EXTERNAL_SOURCE_2) | \
- $(TAR) $(TAR_STRIP_COMPONENTS)=3 --hard-dereference -C $(@D) $(TAR_OPTIONS) -
+ $(HOST_TAR) $(TAR_STRIP_COMPONENTS)=3 --hard-dereference -C $(@D) $(TAR_OPTIONS) -
ifeq ($(TOOLCHAIN_EXTERNAL_PREFIX),bfin-uclinux)
rm -rf $(TOOLCHAIN_EXTERNAL_DIR)/bfin-linux-uclibc
mv $(TOOLCHAIN_EXTERNAL_DIR)/bfin-uclinux $(TOOLCHAIN_EXTERNAL_DIR)/tmp
@@ -250,7 +253,7 @@
$(TOOLCHAIN_EXTERNAL_DIR)/.extracted: $(DL_DIR)/$(TOOLCHAIN_EXTERNAL_SOURCE)
mkdir -p $(@D)
- $(INFLATE$(suffix $(TOOLCHAIN_EXTERNAL_SOURCE))) $^ | $(TAR) $(TAR_STRIP_COMPONENTS)=1 -C $(@D) $(TAR_OPTIONS) -
+ $(INFLATE$(suffix $(TOOLCHAIN_EXTERNAL_SOURCE))) $^ | $(HOST_TAR) $(TAR_STRIP_COMPONENTS)=1 -C $(@D) $(TAR_OPTIONS) -
$(Q)touch $@
endif
^ permalink raw reply [flat|nested] 21+ messages in thread* [Buildroot] [PATCH 5 of 5] toolchain-external: use host-tar instead of tar to unpack toolchains
2011-09-21 6:55 ` [Buildroot] [PATCH 5 of 5] toolchain-external: use host-tar instead of tar to unpack toolchains Thomas De Schampheleire
@ 2011-09-21 8:02 ` Thomas Petazzoni
2011-09-21 8:35 ` Thomas De Schampheleire
0 siblings, 1 reply; 21+ messages in thread
From: Thomas Petazzoni @ 2011-09-21 8:02 UTC (permalink / raw)
To: buildroot
Hello,
Le Wed, 21 Sep 2011 08:55:35 +0200,
Thomas De Schampheleire <patrickdepinguin+buildroot@gmail.com> a ?crit :
> Some toolchains, like the one built with buildroot itself, use hardlinks (for
> example to link between the c++ and g++ binary). Unpacking such a toolchain
> with the --strip-components options does not work correctly if the system tar
> is too old (<1.17). Even recent releases of RedHat/CentOS still ship with
> tar 1.15.
> To avoid such problems, always use host-tar instead of tar to unpack
> toolchains.
I'm sorry but we need to find a better solution for this. I really
don't want all users to pay the price of building tar to extract
the external toolchain just because some crappy distribution has an old
version of tar. Can we make this conditional on the version of tar ?
I'd also like some comment in the code suggesting to remove this crap 1
or 2 years from now, once those RedHat/CentOS distributions will have
been upgraded.
Also, this proposal only takes care of using the compiled host tar for
toolchain extraction. Why don't we also use it to extract all other
packages (except tar itself, of course...) ?
Thomas
--
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com
^ permalink raw reply [flat|nested] 21+ messages in thread
* [Buildroot] [PATCH 5 of 5] toolchain-external: use host-tar instead of tar to unpack toolchains
2011-09-21 8:02 ` Thomas Petazzoni
@ 2011-09-21 8:35 ` Thomas De Schampheleire
2011-09-21 10:54 ` Michael S. Zick
0 siblings, 1 reply; 21+ messages in thread
From: Thomas De Schampheleire @ 2011-09-21 8:35 UTC (permalink / raw)
To: buildroot
Hi,
On Wed, Sep 21, 2011 at 10:02 AM, Thomas Petazzoni
<thomas.petazzoni@free-electrons.com> wrote:
> Hello,
>
> Le Wed, 21 Sep 2011 08:55:35 +0200,
> Thomas De Schampheleire <patrickdepinguin+buildroot@gmail.com> a ?crit :
>
>> Some toolchains, like the one built with buildroot itself, use hardlinks (for
>> example to link between the c++ and g++ binary). Unpacking such a toolchain
>> with the --strip-components options does not work correctly if the system tar
>> is too old (<1.17). Even recent releases of RedHat/CentOS still ship with
>> tar 1.15.
>> To avoid such problems, always use host-tar instead of tar to unpack
>> toolchains.
>
> I'm sorry but we need to find a better solution for this. I really
> don't want all users to pay the price of building tar to extract
> the external toolchain just because some crappy distribution has an old
> version of tar. Can we make this conditional on the version of tar ?
> I'd also like some comment in the code suggesting to remove this crap 1
> or 2 years from now, once those RedHat/CentOS distributions will have
> been upgraded.
>
> Also, this proposal only takes care of using the compiled host tar for
> toolchain extraction. Why don't we also use it to extract all other
> packages (except tar itself, of course...) ?
I agree with both your points. In fact, I raised the same doubts in my
original patches:
http://lists.busybox.net/pipermail/buildroot/2011-August/044784.html
----
To be honest, I'm not sure what the best way is to handle this. The
same problems with tar could occur in other places than just external
toolchains. So maybe we should use host-tar in more places.
On the other hand, most users have a recent, working, tar version on
there system. It may thus make more sense to detect the system tar
version and only use host-tar if it's too old.
----
Regarding deprecating this change after some time: I'm not sure
whether this is still necessary if we make this conditional on the
system tar version. A user that is still on some old corporate system
with RedHat 5, may still need this patch in two or three years.
I'll see if I can update this patch with an automatic detection.
How would you go about to use the host-tar for all packages? I need to
add an extra autotargets call to package/tar/tar.mk, but I also need
to redefine TAR somewhere (currently it's defined in
package/Makefile.package.in). But TAR_STRIP_COMPONENTS uses TAR with a
:= dependency, and package/tar/tar.mk is only included after
package/Makefile.package.in, so I would need to add the conditional
check in two places.
Unless I move the check to a new file that is included from both
tar.mk and Makefile.package.in. In this case, where will I place the
file?
How do you see this?
Thanks for your comments,
Thomas
>
> Thomas
> --
> Thomas Petazzoni, Free Electrons
> Kernel, drivers, real-time and embedded Linux
> development, consulting, training and support.
> http://free-electrons.com
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot
^ permalink raw reply [flat|nested] 21+ messages in thread
* [Buildroot] [PATCH 5 of 5] toolchain-external: use host-tar instead of tar to unpack toolchains
2011-09-21 8:35 ` Thomas De Schampheleire
@ 2011-09-21 10:54 ` Michael S. Zick
2011-09-21 12:16 ` Thomas De Schampheleire
2011-09-22 20:02 ` Arnout Vandecappelle
0 siblings, 2 replies; 21+ messages in thread
From: Michael S. Zick @ 2011-09-21 10:54 UTC (permalink / raw)
To: buildroot
On Wed September 21 2011, Thomas De Schampheleire wrote:
> Hi,
>
> On Wed, Sep 21, 2011 at 10:02 AM, Thomas Petazzoni
> <thomas.petazzoni@free-electrons.com> wrote:
> > Hello,
> >
> > Le Wed, 21 Sep 2011 08:55:35 +0200,
> > Thomas De Schampheleire <patrickdepinguin+buildroot@gmail.com> a ?crit :
> >
> >> Some toolchains, like the one built with buildroot itself, use hardlinks (for
> >> example to link between the c++ and g++ binary). Unpacking such a toolchain
> >> with the --strip-components options does not work correctly if the system tar
> >> is too old (<1.17). Even recent releases of RedHat/CentOS still ship with
> >> tar 1.15.
> >> To avoid such problems, always use host-tar instead of tar to unpack
> >> toolchains.
> >
> > I'm sorry but we need to find a better solution for this. I really
> > don't want all users to pay the price of building tar to extract
> > the external toolchain just because some crappy distribution has an old
> > version of tar. Can we make this conditional on the version of tar ?
> > I'd also like some comment in the code suggesting to remove this crap 1
> > or 2 years from now, once those RedHat/CentOS distributions will have
> > been upgraded.
> >
> > Also, this proposal only takes care of using the compiled host tar for
> > toolchain extraction. Why don't we also use it to extract all other
> > packages (except tar itself, of course...) ?
>
> I agree with both your points.
>
A suggestion here -
In that build system dependencies code/file, the one recently extended
to also list rsync...
Perhaps extend that with a bit of infrastructure that can check not
only if present (with which) but also the version?
That should help keep the build environment sane, with a message
to the user if something is too old;
And should relieve the maintainers of the task of remembering to
"re-correct" the build system someday in the future.
Mike
> In fact, I raised the same doubts in my
> original patches:
> http://lists.busybox.net/pipermail/buildroot/2011-August/044784.html
> ----
> To be honest, I'm not sure what the best way is to handle this. The
> same problems with tar could occur in other places than just external
> toolchains. So maybe we should use host-tar in more places.
> On the other hand, most users have a recent, working, tar version on
> there system. It may thus make more sense to detect the system tar
> version and only use host-tar if it's too old.
> ----
>
>
> Regarding deprecating this change after some time: I'm not sure
> whether this is still necessary if we make this conditional on the
> system tar version. A user that is still on some old corporate system
> with RedHat 5, may still need this patch in two or three years.
>
> I'll see if I can update this patch with an automatic detection.
>
> How would you go about to use the host-tar for all packages? I need to
> add an extra autotargets call to package/tar/tar.mk, but I also need
> to redefine TAR somewhere (currently it's defined in
> package/Makefile.package.in). But TAR_STRIP_COMPONENTS uses TAR with a
> := dependency, and package/tar/tar.mk is only included after
> package/Makefile.package.in, so I would need to add the conditional
> check in two places.
> Unless I move the check to a new file that is included from both
> tar.mk and Makefile.package.in. In this case, where will I place the
> file?
> How do you see this?
>
> Thanks for your comments,
> Thomas
>
> >
> > Thomas
> > --
> > Thomas Petazzoni, Free Electrons
> > Kernel, drivers, real-time and embedded Linux
> > development, consulting, training and support.
> > http://free-electrons.com
> > _______________________________________________
> > buildroot mailing list
> > buildroot at busybox.net
> > http://lists.busybox.net/mailman/listinfo/buildroot
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot
>
>
^ permalink raw reply [flat|nested] 21+ messages in thread
* [Buildroot] [PATCH 5 of 5] toolchain-external: use host-tar instead of tar to unpack toolchains
2011-09-21 10:54 ` Michael S. Zick
@ 2011-09-21 12:16 ` Thomas De Schampheleire
2011-09-22 20:02 ` Arnout Vandecappelle
1 sibling, 0 replies; 21+ messages in thread
From: Thomas De Schampheleire @ 2011-09-21 12:16 UTC (permalink / raw)
To: buildroot
Hi,
On Wed, Sep 21, 2011 at 12:54 PM, Michael S. Zick <minimod@morethan.org> wrote:
> On Wed September 21 2011, Thomas De Schampheleire wrote:
>> Hi,
>>
>> On Wed, Sep 21, 2011 at 10:02 AM, Thomas Petazzoni
>> <thomas.petazzoni@free-electrons.com> wrote:
>> > Hello,
>> >
>> > Le Wed, 21 Sep 2011 08:55:35 +0200,
>> > Thomas De Schampheleire <patrickdepinguin+buildroot@gmail.com> a ?crit :
>> >
>> >> Some toolchains, like the one built with buildroot itself, use hardlinks (for
>> >> example to link between the c++ and g++ binary). Unpacking such a toolchain
>> >> with the --strip-components options does not work correctly if the system tar
>> >> is too old (<1.17). Even recent releases of RedHat/CentOS still ship with
>> >> tar 1.15.
>> >> To avoid such problems, always use host-tar instead of tar to unpack
>> >> toolchains.
>> >
>> > I'm sorry but we need to find a better solution for this. I really
>> > don't want all users to pay the price of building tar to extract
>> > the external toolchain just because some crappy distribution has an old
>> > version of tar. Can we make this conditional on the version of tar ?
>> > I'd also like some comment in the code suggesting to remove this crap 1
>> > or 2 years from now, once those RedHat/CentOS distributions will have
>> > been upgraded.
>> >
>> > Also, this proposal only takes care of using the compiled host tar for
>> > toolchain extraction. Why don't we also use it to extract all other
>> > packages (except tar itself, of course...) ?
>>
>> I agree with both your points.
>>
>
> A suggestion here -
> In that build system dependencies code/file, the one recently extended
> to also list rsync...
>
> Perhaps extend that with a bit of infrastructure that can check not
> only if present (with which) but also the version?
>
> That should help keep the build environment sane, with a message
> to the user if something is too old;
> And should relieve the maintainers of the task of remembering to
> "re-correct" the build system someday in the future.
Thanks for the hint! I will definitely look into that dependencies file..
>
> Mike
>> In fact, I raised the same doubts in my
>> original patches:
>> http://lists.busybox.net/pipermail/buildroot/2011-August/044784.html
>> ----
>> To be honest, I'm not sure what the best way is to handle this. The
>> same problems with tar could occur in other places than just external
>> toolchains. So maybe we should use host-tar in more places.
>> On the other hand, most users have a recent, working, tar version on
>> there system. It may thus make more sense to detect the system tar
>> version and only use host-tar if it's too old.
>> ----
>>
>>
>> Regarding deprecating this change after some time: I'm not sure
>> whether this is still necessary if we make this conditional on the
>> system tar version. A user that is still on some old corporate system
>> with RedHat 5, may still need this patch in two or three years.
>>
>> I'll see if I can update this patch with an automatic detection.
>>
>> How would you go about to use the host-tar for all packages? I need to
>> add an extra autotargets call to package/tar/tar.mk, but I also need
>> to redefine TAR somewhere (currently it's defined in
>> package/Makefile.package.in). But TAR_STRIP_COMPONENTS uses TAR with a
>> := dependency, and package/tar/tar.mk is only included after
>> package/Makefile.package.in, so I would need to add the conditional
>> check in two places.
>> Unless I move the check to a new file that is included from both
>> tar.mk and Makefile.package.in. In this case, where will I place the
>> file?
>> How do you see this?
>>
>> Thanks for your comments,
>> Thomas
>>
>> >
>> > Thomas
>> > --
>> > Thomas Petazzoni, Free Electrons
>> > Kernel, drivers, real-time and embedded Linux
>> > development, consulting, training and support.
>> > http://free-electrons.com
>> > _______________________________________________
>> > buildroot mailing list
>> > buildroot at busybox.net
>> > http://lists.busybox.net/mailman/listinfo/buildroot
>> _______________________________________________
>> buildroot mailing list
>> buildroot at busybox.net
>> http://lists.busybox.net/mailman/listinfo/buildroot
>>
>>
>
>
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot
>
^ permalink raw reply [flat|nested] 21+ messages in thread
* [Buildroot] [PATCH 5 of 5] toolchain-external: use host-tar instead of tar to unpack toolchains
2011-09-21 10:54 ` Michael S. Zick
2011-09-21 12:16 ` Thomas De Schampheleire
@ 2011-09-22 20:02 ` Arnout Vandecappelle
2011-09-23 6:18 ` Thomas De Schampheleire
1 sibling, 1 reply; 21+ messages in thread
From: Arnout Vandecappelle @ 2011-09-22 20:02 UTC (permalink / raw)
To: buildroot
On Wednesday 21 September 2011 12:54:55, Michael S. Zick wrote:
> That should help keep the build environment sane, with a message
> to the user if something is too old;
> And should relieve the maintainers of the task of remembering to
> "re-correct" the build system someday in the future.
How does that help the users who are stuck on RH5? Buildroot would still not
work for them (although with a clearer explanation why not), so they'd have to
implement Thomas's patches after all.
Regards,
Arnout
--
Arnout Vandecappelle arnout at mind be
Senior Embedded Software Architect +32-16-286540
Essensium/Mind http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint: 31BB CF53 8660 6F88 345D 54CC A836 5879 20D7 CF43
^ permalink raw reply [flat|nested] 21+ messages in thread
* [Buildroot] [PATCH 5 of 5] toolchain-external: use host-tar instead of tar to unpack toolchains
2011-09-22 20:02 ` Arnout Vandecappelle
@ 2011-09-23 6:18 ` Thomas De Schampheleire
0 siblings, 0 replies; 21+ messages in thread
From: Thomas De Schampheleire @ 2011-09-23 6:18 UTC (permalink / raw)
To: buildroot
On Thu, Sep 22, 2011 at 10:02 PM, Arnout Vandecappelle <arnout@mind.be> wrote:
>
> On Wednesday 21 September 2011 12:54:55, Michael S. Zick wrote:
>> That should help keep the build environment sane, with a message
>> to the user if something is too old;
>> And should relieve the maintainers of the task of remembering to
>> "re-correct" the build system someday in the future.
>
> ?How does that help the users who are stuck on RH5? ?Buildroot would still not
> work for them (although with a clearer explanation why not), so they'd have to
> implement Thomas's patches after all.
The way I interpreted this was to add a check to dependencies.sh on
the system tar version. If it's too old, we go ahead and build our own
host-tar and use it as $(TAR). If it is recent enough, we don't change
anything.
I still need to try that out though.
Best regards,
Thomas
^ permalink raw reply [flat|nested] 21+ messages in thread