[Buildroot] [PATCH 0 of 6 v2] legal info: split host and package output

[Buildroot] [PATCH 0 of 6 v2] legal info: split host and package output

[Thomas De Schampheleire]

Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>

---
 Makefile                     |  25 +++++++++++++++----------
 docs/manual/legal-notice.txt |  21 ++++++++++++---------
 package/pkg-generic.mk       |  33 +++++++++++++++++++++++++--------
 package/pkg-utils.mk         |  39 ++++++++++++++++++++++++++-------------
 4 files changed, 78 insertions(+), 40 deletions(-)

[Buildroot] [PATCH 1 of 6 v2] legal info: fix saving of host package licenses

[Thomas De Schampheleire]

Due to some tricky make behavior, the license texts of host packages that
did not provide an explicit HOST_FOO_LICENSE_FILES definition was not saved.
The problem is that you cannot correctly use a variable defined/updated
inside a call'ed block as input to a foreach statement. If you try to use
$(FOO) then only the original value of FOO is used for foreach, any update
inside the call'ed block is ignored. However, if you use $$(FOO), the entire
contents of FOO (typically a list of items) is passed as one item to
foreach, thus causing just one iteration instead of several.

To fix the problem, one should only use values in foreach that have not
changed inside the call'ed block. In the case of (HOST_)FOO_LICENSE_FILES,
this means repeating the checks for a valid HOST_FOO_X and using FOO_X as
fallback.

Additionally, a few empty lines have been added to the legal-info-foo block
for clarity, as the amount of nested ifdef/ifeq statements have become very
high.

Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>

---
v2:
- implement a solution for packages that have multiple license files.
- fix accidental mixup of code from another patch

 package/pkg-generic.mk |  23 ++++++++++++++++++++---
 1 files changed, 20 insertions(+), 3 deletions(-)

diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
--- a/package/pkg-generic.mk
+++ b/package/pkg-generic.mk
@@ -499,26 +499,43 @@ endif
 $(1)-legal-info:
 # Packages without a source are assumed to be part of Buildroot, skip them.
 ifneq ($(call qstrip,$$($(2)_SOURCE)),)
+
 ifeq ($$($(2)_SITE_METHOD),local)
 # Packages without a tarball: don't save and warn
 	@$(call legal-warning-pkg-savednothing,$$($(2)_RAWNAME),local)
+
 else ifneq ($$($(2)_OVERRIDE_SRCDIR),)
 	@$(call legal-warning-pkg-savednothing,$$($(2)_RAWNAME),override)
+
 else
 # Other packages
+
 # Save license files if defined
 ifeq ($(call qstrip,$$($(2)_LICENSE_FILES)),)
 	@$(call legal-license-nofiles,$$($(2)_RAWNAME))
 	@$(call legal-warning-pkg,$$($(2)_RAWNAME),cannot save license ($(2)_LICENSE_FILES not defined))
 else
+# We cannot use $$($(2)_LICENSE_FILES) in the foreach below because it fails
+# for host packages that have no explicit HOST_FOO_LICENSE_FILES set. We have
+# to explicitly repeat the check for HOST_FOO_X and FOO_X.
+ifeq ($($(2)_LICENSE_FILES),)
+ifneq ($($(3)_LICENSE_FILES),)
+# Host package with no explicit HOST_FOO_LICENSE_FILES
+	@$(foreach F,$($(3)_LICENSE_FILES),$(call legal-license-file,$$($(2)_RAWNAME),$(F),$$($(2)_DIR)/$(F))$$(sep))
+endif
+else
+# Target package, or host package with explicit HOST_FOO_LICENSE_FILES
 	@$(foreach F,$($(2)_LICENSE_FILES),$(call legal-license-file,$$($(2)_RAWNAME),$(F),$$($(2)_DIR)/$(F))$$(sep))
-endif
+endif # host/target distinction
+endif # license files
+
 ifeq ($$($(2)_REDISTRIBUTE),YES)
 # Copy the source tarball (just hardlink if possible)
 	@cp -l $(DL_DIR)/$$($(2)_SOURCE) $(REDIST_SOURCES_DIR) 2>/dev/null || \
 	   cp $(DL_DIR)/$$($(2)_SOURCE) $(REDIST_SOURCES_DIR)
-endif
-endif
+endif # redistribute
+
+endif # other packages
 	@$(call legal-manifest,$$($(2)_RAWNAME),$$($(2)_VERSION),$$($(2)_LICENSE),$$($(2)_MANIFEST_LICENSE_FILES),$$($(2)_MANIFEST_TARBALL))
 endif # ifneq ($(call qstrip,$$($(2)_SOURCE)),)
 	$(foreach hook,$($(2)_POST_LEGAL_INFO_HOOKS),$(call $(hook))$(sep))

[Buildroot] [PATCH 2 of 6 v2] legal info: split manifest for host and target

[Thomas De Schampheleire]

Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Acked-by: Luca Ceresoli <luca@lucaceresoli.net>
Tested-by: Luca Ceresoli <luca@lucaceresoli.net>

---
v2:
- trivial rebase after changes in first patch
- move out documentation to a separate patch

 Makefile               |  8 +++++---
 package/pkg-generic.mk |  2 +-
 package/pkg-utils.mk   |  2 +-
 3 files changed, 7 insertions(+), 5 deletions(-)

diff --git a/Makefile b/Makefile
--- a/Makefile
+++ b/Makefile
@@ -110,7 +110,8 @@ HOST_DIR:=$(BASE_DIR)/host
 LEGAL_INFO_DIR=$(BASE_DIR)/legal-info
 REDIST_SOURCES_DIR=$(LEGAL_INFO_DIR)/sources
 LICENSE_FILES_DIR=$(LEGAL_INFO_DIR)/licenses
-LEGAL_MANIFEST_CSV=$(LEGAL_INFO_DIR)/manifest.csv
+LEGAL_MANIFEST_CSV_TARGET=$(LEGAL_INFO_DIR)/manifest.csv
+LEGAL_MANIFEST_CSV_HOST=$(LEGAL_INFO_DIR)/host-manifest.csv
 LEGAL_LICENSES_TXT=$(LEGAL_INFO_DIR)/licenses.txt
 LEGAL_WARNINGS=$(LEGAL_INFO_DIR)/.warnings
 LEGAL_REPORT=$(LEGAL_INFO_DIR)/README
@@ -588,8 +589,9 @@ legal-info-clean:
 legal-info-prepare: $(LEGAL_INFO_DIR)
 	@$(call MESSAGE,"Collecting legal info")
 	@$(call legal-license-file,buildroot,COPYING,COPYING)
-	@$(call legal-manifest,PACKAGE,VERSION,LICENSE,LICENSE FILES,SOURCE ARCHIVE)
-	@$(call legal-manifest,buildroot,$(BR2_VERSION_FULL),GPLv2+,COPYING,not saved)
+	@$(call legal-manifest,PACKAGE,VERSION,LICENSE,LICENSE FILES,SOURCE ARCHIVE,TARGET)
+	@$(call legal-manifest,PACKAGE,VERSION,LICENSE,LICENSE FILES,SOURCE ARCHIVE,HOST)
+	@$(call legal-manifest,buildroot,$(BR2_VERSION_FULL),GPLv2+,COPYING,not saved,HOST)
 	@$(call legal-warning,the Buildroot source code has not been saved)
 	@$(call legal-warning,the toolchain has not been saved)
 	@cp $(BUILDROOT_CONFIG) $(LEGAL_INFO_DIR)/buildroot.config
diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
--- a/package/pkg-generic.mk
+++ b/package/pkg-generic.mk
@@ -536,7 +536,7 @@ ifeq ($$($(2)_REDISTRIBUTE),YES)
 endif # redistribute
 
 endif # other packages
-	@$(call legal-manifest,$$($(2)_RAWNAME),$$($(2)_VERSION),$$($(2)_LICENSE),$$($(2)_MANIFEST_LICENSE_FILES),$$($(2)_MANIFEST_TARBALL))
+	@$(call legal-manifest,$$($(2)_RAWNAME),$$($(2)_VERSION),$$($(2)_LICENSE),$$($(2)_MANIFEST_LICENSE_FILES),$$($(2)_MANIFEST_TARBALL),$(call UPPERCASE,$(5)))
 endif # ifneq ($(call qstrip,$$($(2)_SOURCE)),)
 	$(foreach hook,$($(2)_POST_LEGAL_INFO_HOOKS),$(call $(hook))$(sep))
 
diff --git a/package/pkg-utils.mk b/package/pkg-utils.mk
--- a/package/pkg-utils.mk
+++ b/package/pkg-utils.mk
@@ -96,7 +96,7 @@ legal-warning-pkg=echo "WARNING: $(1): $
 define legal-warning-pkg-savednothing # pkg, {local|override}
 	$(call legal-warning-pkg,$(1),sources and license files not saved ($(2) packages not handled))
 endef
-legal-manifest=echo '"$(1)","$(2)","$(3)","$(4)","$(5)"' >>$(LEGAL_MANIFEST_CSV)
+legal-manifest=echo '"$(1)","$(2)","$(3)","$(4)","$(5)"' >>$(LEGAL_MANIFEST_CSV_$(6))
 define legal-license-header
 	echo -e "$(LEGAL_INFO_SEPARATOR)\n\t$(1):" \
 		"$(2)\n$(LEGAL_INFO_SEPARATOR)\n\n" >>$(LEGAL_LICENSES_TXT)

[Buildroot] [PATCH 3 of 6 v2] legal info: split license texts for host and target

[Thomas De Schampheleire]

Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Acked-by: Luca Ceresoli <luca@lucaceresoli.net>

---
v2:
- rebase after changes in first patch
- fix accidental mixup of contents from first patch

 Makefile               |   8 +++++---
 package/pkg-generic.mk |   6 +++---
 package/pkg-utils.mk   |  16 ++++++++--------
 3 files changed, 16 insertions(+), 14 deletions(-)

diff --git a/Makefile b/Makefile
--- a/Makefile
+++ b/Makefile
@@ -109,10 +109,12 @@ HOST_DIR:=$(BASE_DIR)/host
 
 LEGAL_INFO_DIR=$(BASE_DIR)/legal-info
 REDIST_SOURCES_DIR=$(LEGAL_INFO_DIR)/sources
-LICENSE_FILES_DIR=$(LEGAL_INFO_DIR)/licenses
+LICENSE_FILES_DIR_TARGET=$(LEGAL_INFO_DIR)/licenses
+LICENSE_FILES_DIR_HOST=$(LEGAL_INFO_DIR)/host-licenses
 LEGAL_MANIFEST_CSV_TARGET=$(LEGAL_INFO_DIR)/manifest.csv
 LEGAL_MANIFEST_CSV_HOST=$(LEGAL_INFO_DIR)/host-manifest.csv
-LEGAL_LICENSES_TXT=$(LEGAL_INFO_DIR)/licenses.txt
+LEGAL_LICENSES_TXT_TARGET=$(LEGAL_INFO_DIR)/licenses.txt
+LEGAL_LICENSES_TXT_HOST=$(LEGAL_INFO_DIR)/host-licenses.txt
 LEGAL_WARNINGS=$(LEGAL_INFO_DIR)/.warnings
 LEGAL_REPORT=$(LEGAL_INFO_DIR)/README
 
@@ -588,7 +590,7 @@ legal-info-clean:
 
 legal-info-prepare: $(LEGAL_INFO_DIR)
 	@$(call MESSAGE,"Collecting legal info")
-	@$(call legal-license-file,buildroot,COPYING,COPYING)
+	@$(call legal-license-file,buildroot,COPYING,COPYING,HOST)
 	@$(call legal-manifest,PACKAGE,VERSION,LICENSE,LICENSE FILES,SOURCE ARCHIVE,TARGET)
 	@$(call legal-manifest,PACKAGE,VERSION,LICENSE,LICENSE FILES,SOURCE ARCHIVE,HOST)
 	@$(call legal-manifest,buildroot,$(BR2_VERSION_FULL),GPLv2+,COPYING,not saved,HOST)
diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
--- a/package/pkg-generic.mk
+++ b/package/pkg-generic.mk
@@ -512,7 +512,7 @@ else
 
 # Save license files if defined
 ifeq ($(call qstrip,$$($(2)_LICENSE_FILES)),)
-	@$(call legal-license-nofiles,$$($(2)_RAWNAME))
+	@$(call legal-license-nofiles,$$($(2)_RAWNAME),$(call UPPERCASE,$(5)))
 	@$(call legal-warning-pkg,$$($(2)_RAWNAME),cannot save license ($(2)_LICENSE_FILES not defined))
 else
 # We cannot use $$($(2)_LICENSE_FILES) in the foreach below because it fails
@@ -521,11 +521,11 @@ else
 ifeq ($($(2)_LICENSE_FILES),)
 ifneq ($($(3)_LICENSE_FILES),)
 # Host package with no explicit HOST_FOO_LICENSE_FILES
-	@$(foreach F,$($(3)_LICENSE_FILES),$(call legal-license-file,$$($(2)_RAWNAME),$(F),$$($(2)_DIR)/$(F))$$(sep))
+	@$(foreach F,$($(3)_LICENSE_FILES),$(call legal-license-file,$$($(2)_RAWNAME),$(F),$$($(2)_DIR)/$(F),$(call UPPERCASE,$(5)))$$(sep))
 endif
 else
 # Target package, or host package with explicit HOST_FOO_LICENSE_FILES
-	@$(foreach F,$($(2)_LICENSE_FILES),$(call legal-license-file,$$($(2)_RAWNAME),$(F),$$($(2)_DIR)/$(F))$$(sep))
+	@$(foreach F,$($(2)_LICENSE_FILES),$(call legal-license-file,$$($(2)_RAWNAME),$(F),$$($(2)_DIR)/$(F),$(call UPPERCASE,$(5)))$$(sep))
 endif # host/target distinction
 endif # license files
 
diff --git a/package/pkg-utils.mk b/package/pkg-utils.mk
--- a/package/pkg-utils.mk
+++ b/package/pkg-utils.mk
@@ -99,15 +99,15 @@ endef
 legal-manifest=echo '"$(1)","$(2)","$(3)","$(4)","$(5)"' >>$(LEGAL_MANIFEST_CSV_$(6))
 define legal-license-header
 	echo -e "$(LEGAL_INFO_SEPARATOR)\n\t$(1):" \
-		"$(2)\n$(LEGAL_INFO_SEPARATOR)\n\n" >>$(LEGAL_LICENSES_TXT)
+		"$(2)\n$(LEGAL_INFO_SEPARATOR)\n\n" >>$(LEGAL_LICENSES_TXT_$(3))
 endef
 define legal-license-nofiles
-	$(call legal-license-header,$(1),unknown license file(s))
+	$(call legal-license-header,$(1),unknown license file(s),$(2))
 endef
-define legal-license-file # pkg, filename, file-fullpath
-	$(call legal-license-header,$(1),$(2) file) && \
-	cat $(3) >>$(LEGAL_LICENSES_TXT) && \
-	echo >>$(LEGAL_LICENSES_TXT) && \
-	mkdir -p $(LICENSE_FILES_DIR)/$(1)/$(dir $(2)) && \
-	cp $(3) $(LICENSE_FILES_DIR)/$(1)/$(2)
+define legal-license-file # pkg, filename, file-fullpath, type
+	$(call legal-license-header,$(1),$(2) file,$(4)) && \
+	cat $(3) >>$(LEGAL_LICENSES_TXT_$(4)) && \
+	echo >>$(LEGAL_LICENSES_TXT_$(4)) && \
+	mkdir -p $(LICENSE_FILES_DIR_$(4))/$(1)/$(dir $(2)) && \
+	cp $(3) $(LICENSE_FILES_DIR_$(4))/$(1)/$(2)
 endef

[Buildroot] [PATCH 4 of 6 v2] legal info: split sources for host and target

[Thomas De Schampheleire]

Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Acked-by: Luca Ceresoli <luca@lucaceresoli.net>

---
This patch concludes the splitting of legal info for host and target.

v2: trivial rebase after changes in first patch

 Makefile               |  9 +++++----
 package/pkg-generic.mk |  6 +++---
 2 files changed, 8 insertions(+), 7 deletions(-)

diff --git a/Makefile b/Makefile
--- a/Makefile
+++ b/Makefile
@@ -108,7 +108,8 @@ TARGET_DIR:=$(BASE_DIR)/target
 HOST_DIR:=$(BASE_DIR)/host
 
 LEGAL_INFO_DIR=$(BASE_DIR)/legal-info
-REDIST_SOURCES_DIR=$(LEGAL_INFO_DIR)/sources
+REDIST_SOURCES_DIR_TARGET=$(LEGAL_INFO_DIR)/sources
+REDIST_SOURCES_DIR_HOST=$(LEGAL_INFO_DIR)/host-sources
 LICENSE_FILES_DIR_TARGET=$(LEGAL_INFO_DIR)/licenses
 LICENSE_FILES_DIR_HOST=$(LEGAL_INFO_DIR)/host-licenses
 LEGAL_MANIFEST_CSV_TARGET=$(LEGAL_INFO_DIR)/manifest.csv
@@ -419,7 +420,7 @@ world: toolchain $(TARGETS_ALL)
 # dependencies anywhere else
 #
 ################################################################################
-$(BUILD_DIR) $(HOST_DIR) $(BINARIES_DIR) $(STAMP_DIR) $(LEGAL_INFO_DIR) $(REDIST_SOURCES_DIR):
+$(BUILD_DIR) $(HOST_DIR) $(BINARIES_DIR) $(STAMP_DIR) $(LEGAL_INFO_DIR) $(REDIST_SOURCES_DIR_TARGET) $(REDIST_SOURCES_DIR_HOST):
 	@mkdir -p $@
 
 # We make a symlink lib32->lib or lib64->lib as appropriate
@@ -598,8 +599,8 @@ legal-info-prepare: $(LEGAL_INFO_DIR)
 	@$(call legal-warning,the toolchain has not been saved)
 	@cp $(BUILDROOT_CONFIG) $(LEGAL_INFO_DIR)/buildroot.config
 
-legal-info: dirs legal-info-clean legal-info-prepare $(REDIST_SOURCES_DIR) \
-		$(TARGETS_LEGAL_INFO)
+legal-info: dirs legal-info-clean legal-info-prepare $(TARGETS_LEGAL_INFO) \
+		$(REDIST_SOURCES_DIR_TARGET) $(REDIST_SOURCES_DIR_HOST)
 	@cat support/legal-info/README.header >>$(LEGAL_REPORT)
 	@if [ -r $(LEGAL_WARNINGS) ]; then \
 		cat support/legal-info/README.warnings-header \
diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
--- a/package/pkg-generic.mk
+++ b/package/pkg-generic.mk
@@ -488,7 +488,7 @@ ifeq ($$($(2)_REDISTRIBUTE),YES)
 ifneq ($$($(2)_SITE_METHOD),local)
 ifneq ($$($(2)_SITE_METHOD),override)
 # Packages that have a tarball need it downloaded and extracted beforehand
-$(1)-legal-info: $(1)-extract $(REDIST_SOURCES_DIR)
+$(1)-legal-info: $(1)-extract $(REDIST_SOURCES_DIR_$(call UPPERCASE,$(5)))
 $(2)_MANIFEST_TARBALL = $$($(2)_SOURCE)
 endif
 endif
@@ -531,8 +531,8 @@ endif # license files
 
 ifeq ($$($(2)_REDISTRIBUTE),YES)
 # Copy the source tarball (just hardlink if possible)
-	@cp -l $(DL_DIR)/$$($(2)_SOURCE) $(REDIST_SOURCES_DIR) 2>/dev/null || \
-	   cp $(DL_DIR)/$$($(2)_SOURCE) $(REDIST_SOURCES_DIR)
+	@cp -l $(DL_DIR)/$$($(2)_SOURCE) $(REDIST_SOURCES_DIR_$(call UPPERCASE,$(5))) 2>/dev/null || \
+	   cp $(DL_DIR)/$$($(2)_SOURCE) $(REDIST_SOURCES_DIR_$(call UPPERCASE,$(5)))
 endif # redistribute
 
 endif # other packages

[Buildroot] [PATCH 5 of 6 v2] legal info: cleanup utility functions

[Thomas De Schampheleire]

The legal-info utility functions where defined using two ways
util-foo = command-foo
and
define util-bar # parameter description
	command-bar
endef

This commit changes these functions to use the second form for clarity and
additionally adds parameter descriptions on all functions.

Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Acked-by: Luca Ceresoli <luca@lucaceresoli.net>

---
v2:
- add whitespace between functions
- replace 'type' with {HOST|TARGET} (comment Luca)

 package/pkg-utils.mk |  25 +++++++++++++++++++------
 1 files changed, 19 insertions(+), 6 deletions(-)

diff --git a/package/pkg-utils.mk b/package/pkg-utils.mk
--- a/package/pkg-utils.mk
+++ b/package/pkg-utils.mk
@@ -91,20 +91,33 @@ endef
 # legal-info helper functions
 #
 LEGAL_INFO_SEPARATOR="::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::"
-legal-warning=echo "WARNING: $(1)" >>$(LEGAL_WARNINGS)
-legal-warning-pkg=echo "WARNING: $(1): $(2)" >>$(LEGAL_WARNINGS)
+
+define legal-warning # text
+	echo "WARNING: $(1)" >>$(LEGAL_WARNINGS)
+endef
+
+define legal-warning-pkg # pkg, text
+	echo "WARNING: $(1): $(2)" >>$(LEGAL_WARNINGS)
+endef
+
 define legal-warning-pkg-savednothing # pkg, {local|override}
 	$(call legal-warning-pkg,$(1),sources and license files not saved ($(2) packages not handled))
 endef
-legal-manifest=echo '"$(1)","$(2)","$(3)","$(4)","$(5)"' >>$(LEGAL_MANIFEST_CSV_$(6))
-define legal-license-header
+
+define legal-manifest # pkg, version, license, license-files, source, {HOST|TARGET}
+	echo '"$(1)","$(2)","$(3)","$(4)","$(5)"' >>$(LEGAL_MANIFEST_CSV_$(6))
+endef
+
+define legal-license-header # pkg, license-file, {HOST|TARGET}
 	echo -e "$(LEGAL_INFO_SEPARATOR)\n\t$(1):" \
 		"$(2)\n$(LEGAL_INFO_SEPARATOR)\n\n" >>$(LEGAL_LICENSES_TXT_$(3))
 endef
-define legal-license-nofiles
+
+define legal-license-nofiles # pkg, {HOST|TARGET}
 	$(call legal-license-header,$(1),unknown license file(s),$(2))
 endef
-define legal-license-file # pkg, filename, file-fullpath, type
+
+define legal-license-file # pkg, filename, file-fullpath, {HOST|TARGET}
 	$(call legal-license-header,$(1),$(2) file,$(4)) && \
 	cat $(3) >>$(LEGAL_LICENSES_TXT_$(4)) && \
 	echo >>$(LEGAL_LICENSES_TXT_$(4)) && \

[Buildroot] [PATCH 6 of 6 v2] legal info: update documentation with split target/host output

[Thomas De Schampheleire]

Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>

---
v2: new patch

 docs/manual/legal-notice.txt |  15 +++++++++------
 1 files changed, 9 insertions(+), 6 deletions(-)

diff --git a/docs/manual/legal-notice.txt b/docs/manual/legal-notice.txt
--- a/docs/manual/legal-notice.txt
+++ b/docs/manual/legal-notice.txt
@@ -39,16 +39,19 @@ There you will find:
 * +buildroot.config+: this is the Buildroot configuration file that is usually
   produced with +make menuconfig+, and which is necessary to reproduce the
   build.
-* The source code for all packages; this is saved in the +sources/+
-  subdirectory (except for proprietary packages, whose source code is not
-  saved);
-  patches applied to some packages by Buildroot are distributed with the
-  Buildroot sources and are not duplicated in the +sources/+ subdirectory.
-* A manifest file listing the configured packages, their version, license and
-  related information.
+* The source code for all packages; this is saved in the +sources/+ and
+  +host-sources/+ subdirectories for target and host packages respectively.
+  The source code for packages that set +<PKG>_REDISTRIBUTE = NO+ will not be
+  saved.
+  Patches applied to some packages by Buildroot are distributed with the
+  Buildroot sources and are not duplicated in the +sources/+ and +host-sources/+
+  subdirectories.
+* A manifest file (one for host and one for target packages) listing the
+  configured packages, their version, license and related information.
   Some of this information might not be defined in Buildroot; such items are
   marked as "unknown".
-* A +licenses/+ subdirectory, which contains the license text of packages.
+* The license texts of all packages, in the +licenses/+ and +host-licenses/+
+  subdirectories for target and host packages respectively.
   If the license file(s) are not defined in Buildroot, the file is not produced
   and a warning in the +README+ indicates this.
 
@@ -72,7 +75,7 @@ License abbreviations
 ---------------------
 
 Here is a list of the licenses that are most widely used by packages in
-Buildroot, with the name used in the manifest file:
+Buildroot, with the name used in the manifest files:
 
 * `GPLv2`:
   http://www.gnu.org/licenses/old-licenses/gpl-2.0.html[

[Buildroot] [PATCH 1 of 6 v2] legal info: fix saving of host package licenses

[Thomas De Schampheleire]

On Mon, Oct 7, 2013 at 3:15 PM, Thomas De Schampheleire
<patrickdepinguin@gmail.com> wrote:
> Due to some tricky make behavior, the license texts of host packages that
> did not provide an explicit HOST_FOO_LICENSE_FILES definition was not saved.
> The problem is that you cannot correctly use a variable defined/updated
> inside a call'ed block as input to a foreach statement. If you try to use
> $(FOO) then only the original value of FOO is used for foreach, any update
> inside the call'ed block is ignored. However, if you use $$(FOO), the entire
> contents of FOO (typically a list of items) is passed as one item to
> foreach, thus causing just one iteration instead of several.
>
> To fix the problem, one should only use values in foreach that have not
> changed inside the call'ed block. In the case of (HOST_)FOO_LICENSE_FILES,
> this means repeating the checks for a valid HOST_FOO_X and using FOO_X as
> fallback.
>
> Additionally, a few empty lines have been added to the legal-info-foo block
> for clarity, as the amount of nested ifdef/ifeq statements have become very
> high.
>
> Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
>
> ---
> v2:
> - implement a solution for packages that have multiple license files.
> - fix accidental mixup of code from another patch
>
>  package/pkg-generic.mk |  23 ++++++++++++++++++++---
>  1 files changed, 20 insertions(+), 3 deletions(-)
>
> diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
> --- a/package/pkg-generic.mk
> +++ b/package/pkg-generic.mk
> @@ -499,26 +499,43 @@ endif
>  $(1)-legal-info:
>  # Packages without a source are assumed to be part of Buildroot, skip them.
>  ifneq ($(call qstrip,$$($(2)_SOURCE)),)
> +
>  ifeq ($$($(2)_SITE_METHOD),local)
>  # Packages without a tarball: don't save and warn
>         @$(call legal-warning-pkg-savednothing,$$($(2)_RAWNAME),local)
> +
>  else ifneq ($$($(2)_OVERRIDE_SRCDIR),)
>         @$(call legal-warning-pkg-savednothing,$$($(2)_RAWNAME),override)
> +
>  else
>  # Other packages
> +
>  # Save license files if defined
>  ifeq ($(call qstrip,$$($(2)_LICENSE_FILES)),)
>         @$(call legal-license-nofiles,$$($(2)_RAWNAME))
>         @$(call legal-warning-pkg,$$($(2)_RAWNAME),cannot save license ($(2)_LICENSE_FILES not defined))
>  else
> +# We cannot use $$($(2)_LICENSE_FILES) in the foreach below because it fails
> +# for host packages that have no explicit HOST_FOO_LICENSE_FILES set. We have
> +# to explicitly repeat the check for HOST_FOO_X and FOO_X.
> +ifeq ($($(2)_LICENSE_FILES),)
> +ifneq ($($(3)_LICENSE_FILES),)
> +# Host package with no explicit HOST_FOO_LICENSE_FILES
> +       @$(foreach F,$($(3)_LICENSE_FILES),$(call legal-license-file,$$($(2)_RAWNAME),$(F),$$($(2)_DIR)/$(F))$$(sep))
> +endif
> +else
> +# Target package, or host package with explicit HOST_FOO_LICENSE_FILES
>         @$(foreach F,$($(2)_LICENSE_FILES),$(call legal-license-file,$$($(2)_RAWNAME),$(F),$$($(2)_DIR)/$(F))$$(sep))
> -endif
> +endif # host/target distinction
> +endif # license files
> +
>  ifeq ($$($(2)_REDISTRIBUTE),YES)
>  # Copy the source tarball (just hardlink if possible)
>         @cp -l $(DL_DIR)/$$($(2)_SOURCE) $(REDIST_SOURCES_DIR) 2>/dev/null || \
>            cp $(DL_DIR)/$$($(2)_SOURCE) $(REDIST_SOURCES_DIR)
> -endif
> -endif
> +endif # redistribute
> +
> +endif # other packages
>         @$(call legal-manifest,$$($(2)_RAWNAME),$$($(2)_VERSION),$$($(2)_LICENSE),$$($(2)_MANIFEST_LICENSE_FILES),$$($(2)_MANIFEST_TARBALL))
>  endif # ifneq ($(call qstrip,$$($(2)_SOURCE)),)
>         $(foreach hook,$($(2)_POST_LEGAL_INFO_HOOKS),$(call $(hook))$(sep))


ping?

[Buildroot] [PATCH 6 of 6 v2] legal info: update documentation with split target/host output

[Thomas De Schampheleire]

On Mon, Oct 7, 2013 at 3:15 PM, Thomas De Schampheleire
<patrickdepinguin@gmail.com> wrote:
> Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
>
> ---
> v2: new patch
>
>  docs/manual/legal-notice.txt |  15 +++++++++------
>  1 files changed, 9 insertions(+), 6 deletions(-)
>
> diff --git a/docs/manual/legal-notice.txt b/docs/manual/legal-notice.txt
> --- a/docs/manual/legal-notice.txt
> +++ b/docs/manual/legal-notice.txt
> @@ -39,16 +39,19 @@ There you will find:
>  * +buildroot.config+: this is the Buildroot configuration file that is usually
>    produced with +make menuconfig+, and which is necessary to reproduce the
>    build.
> -* The source code for all packages; this is saved in the +sources/+
> -  subdirectory (except for proprietary packages, whose source code is not
> -  saved);
> -  patches applied to some packages by Buildroot are distributed with the
> -  Buildroot sources and are not duplicated in the +sources/+ subdirectory.
> -* A manifest file listing the configured packages, their version, license and
> -  related information.
> +* The source code for all packages; this is saved in the +sources/+ and
> +  +host-sources/+ subdirectories for target and host packages respectively.
> +  The source code for packages that set +<PKG>_REDISTRIBUTE = NO+ will not be
> +  saved.
> +  Patches applied to some packages by Buildroot are distributed with the
> +  Buildroot sources and are not duplicated in the +sources/+ and +host-sources/+
> +  subdirectories.
> +* A manifest file (one for host and one for target packages) listing the
> +  configured packages, their version, license and related information.
>    Some of this information might not be defined in Buildroot; such items are
>    marked as "unknown".
> -* A +licenses/+ subdirectory, which contains the license text of packages.
> +* The license texts of all packages, in the +licenses/+ and +host-licenses/+
> +  subdirectories for target and host packages respectively.
>    If the license file(s) are not defined in Buildroot, the file is not produced
>    and a warning in the +README+ indicates this.
>
> @@ -72,7 +75,7 @@ License abbreviations
>  ---------------------
>
>  Here is a list of the licenses that are most widely used by packages in
> -Buildroot, with the name used in the manifest file:
> +Buildroot, with the name used in the manifest files:
>
>  * `GPLv2`:
>    http://www.gnu.org/licenses/old-licenses/gpl-2.0.html[

This patch, together with the first one in this series, are the only
ones that still need a review, all others have been Acked by Luca.

It would be great if this could get in 2013.11...

Thanks a lot,
Thomas

[Buildroot] [PATCH 6 of 6 v2] legal info: update documentation with split target/host output

[Samuel Martin]

2013/11/11 Thomas De Schampheleire <patrickdepinguin@gmail.com>

> On Mon, Oct 7, 2013 at 3:15 PM, Thomas De Schampheleire
> <patrickdepinguin@gmail.com> wrote:
> > Signed-off-by: Thomas De Schampheleire <
> thomas.de.schampheleire at gmail.com>
>
Acked-by: Samuel Martin <s.martin49@gmail.com>

>
> > ---
> > v2: new patch
> >
> >  docs/manual/legal-notice.txt |  15 +++++++++------
> >  1 files changed, 9 insertions(+), 6 deletions(-)
> >
> > diff --git a/docs/manual/legal-notice.txt b/docs/manual/legal-notice.txt
> > --- a/docs/manual/legal-notice.txt
> > +++ b/docs/manual/legal-notice.txt
> > @@ -39,16 +39,19 @@ There you will find:
> >  * +buildroot.config+: this is the Buildroot configuration file that is
> usually
> >    produced with +make menuconfig+, and which is necessary to reproduce
> the
> >    build.
> > -* The source code for all packages; this is saved in the +sources/+
> > -  subdirectory (except for proprietary packages, whose source code is
> not
> > -  saved);
> > -  patches applied to some packages by Buildroot are distributed with the
> > -  Buildroot sources and are not duplicated in the +sources/+
> subdirectory.
> > -* A manifest file listing the configured packages, their version,
> license and
> > -  related information.
> > +* The source code for all packages; this is saved in the +sources/+ and
> > +  +host-sources/+ subdirectories for target and host packages
> respectively.
> > +  The source code for packages that set +<PKG>_REDISTRIBUTE = NO+ will
> not be
> > +  saved.
> > +  Patches applied to some packages by Buildroot are distributed with the
> > +  Buildroot sources and are not duplicated in the +sources/+ and
> +host-sources/+
> > +  subdirectories.
> > +* A manifest file (one for host and one for target packages) listing the
> > +  configured packages, their version, license and related information.
> >    Some of this information might not be defined in Buildroot; such
> items are
> >    marked as "unknown".
> > -* A +licenses/+ subdirectory, which contains the license text of
> packages.
> > +* The license texts of all packages, in the +licenses/+ and
> +host-licenses/+
> > +  subdirectories for target and host packages respectively.
> >    If the license file(s) are not defined in Buildroot, the file is not
> produced
> >    and a warning in the +README+ indicates this.
> >
> > @@ -72,7 +75,7 @@ License abbreviations
> >  ---------------------
> >
> >  Here is a list of the licenses that are most widely used by packages in
> > -Buildroot, with the name used in the manifest file:
> > +Buildroot, with the name used in the manifest files:
> >
> >  * `GPLv2`:
> >    http://www.gnu.org/licenses/old-licenses/gpl-2.0.html[
>
> This patch, together with the first one in this series, are the only
> ones that still need a review, all others have been Acked by Luca.
>
> It would be great if this could get in 2013.11...
>
> Thanks a lot,
> Thomas
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot
>



-- 
Samuel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20131111/9d591251/attachment.html>

[Buildroot] [PATCH 1 of 6 v2] legal info: fix saving of host package licenses

[Luca Ceresoli]

Thomas, All,

Thomas De Schampheleire wrote:
> Due to some tricky make behavior, the license texts of host packages that
> did not provide an explicit HOST_FOO_LICENSE_FILES definition was not saved.
> The problem is that you cannot correctly use a variable defined/updated
> inside a call'ed block as input to a foreach statement. If you try to use
> $(FOO) then only the original value of FOO is used for foreach, any update
> inside the call'ed block is ignored. However, if you use $$(FOO), the entire
> contents of FOO (typically a list of items) is passed as one item to
> foreach, thus causing just one iteration instead of several.
>
> To fix the problem, one should only use values in foreach that have not
> changed inside the call'ed block. In the case of (HOST_)FOO_LICENSE_FILES,
> this means repeating the checks for a valid HOST_FOO_X and using FOO_X as
> fallback.
>
> Additionally, a few empty lines have been added to the legal-info-foo block
> for clarity, as the amount of nested ifdef/ifeq statements have become very
> high.
>
> Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>

It is not clear to me whether this is a bug in make or something 
"intended". Do you have any more insight?

In either case, it's great you noticed and fixed it, thanks!

Acked-by: Luca Ceresoli <luca@lucaceresoli.net>
Tested-by: Luca Ceresoli <luca@lucaceresoli.net>

-- 
Luca

[Buildroot] [PATCH 1 of 6 v2] legal info: fix saving of host package licenses

[Arnout Vandecappelle]

On 07/10/13 15:15, Thomas De Schampheleire wrote:
> Due to some tricky make behavior, the license texts of host packages that
> did not provide an explicit HOST_FOO_LICENSE_FILES definition was not saved.
> The problem is that you cannot correctly use a variable defined/updated
> inside a call'ed block as input to a foreach statement. If you try to use
> $(FOO) then only the original value of FOO is used for foreach, any update
> inside the call'ed block is ignored. However, if you use $$(FOO), the entire
> contents of FOO (typically a list of items) is passed as one item to
> foreach, thus causing just one iteration instead of several.

 The description of the problem is wrong... It has nothing to do with call,
it is due to the eval behaviour.

 Any variable referenced with a single $ inside the inner-generic-package macro
is expanded before the resulting contents are eval'ed. Therefore, it is not
possible to refer to variables defined by the inner-generic-package macro
from within a single-$ function call.

 Fortunately there is a simple solution: use $$. See below...

> 
> To fix the problem, one should only use values in foreach that have not
> changed inside the call'ed block. In the case of (HOST_)FOO_LICENSE_FILES,
> this means repeating the checks for a valid HOST_FOO_X and using FOO_X as
> fallback.
> 
> Additionally, a few empty lines have been added to the legal-info-foo block
> for clarity, as the amount of nested ifdef/ifeq statements have become very
> high.
> 
> Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
> 
> ---
> v2:
> - implement a solution for packages that have multiple license files.
> - fix accidental mixup of code from another patch
> 
>   package/pkg-generic.mk |  23 ++++++++++++++++++++---
>   1 files changed, 20 insertions(+), 3 deletions(-)
> 
> diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
> --- a/package/pkg-generic.mk
> +++ b/package/pkg-generic.mk
> @@ -499,26 +499,43 @@ endif
>   $(1)-legal-info:
>   # Packages without a source are assumed to be part of Buildroot, skip them.
>   ifneq ($(call qstrip,$$($(2)_SOURCE)),)
> +
>   ifeq ($$($(2)_SITE_METHOD),local)
>   # Packages without a tarball: don't save and warn
>   	@$(call legal-warning-pkg-savednothing,$$($(2)_RAWNAME),local)
> +
>   else ifneq ($$($(2)_OVERRIDE_SRCDIR),)
>   	@$(call legal-warning-pkg-savednothing,$$($(2)_RAWNAME),override)
> +
>   else
>   # Other packages
> +
>   # Save license files if defined
>   ifeq ($(call qstrip,$$($(2)_LICENSE_FILES)),)
>   	@$(call legal-license-nofiles,$$($(2)_RAWNAME))
>   	@$(call legal-warning-pkg,$$($(2)_RAWNAME),cannot save license ($(2)_LICENSE_FILES not defined))
>   else
> +# We cannot use $$($(2)_LICENSE_FILES) in the foreach below because it fails
> +# for host packages that have no explicit HOST_FOO_LICENSE_FILES set. We have
> +# to explicitly repeat the check for HOST_FOO_X and FOO_X.
> +ifeq ($($(2)_LICENSE_FILES),)
> +ifneq ($($(3)_LICENSE_FILES),)
> +# Host package with no explicit HOST_FOO_LICENSE_FILES
> +	@$(foreach F,$($(3)_LICENSE_FILES),$(call legal-license-file,$$($(2)_RAWNAME),$(F),$$($(2)_DIR)/$(F))$$(sep))
> +endif
> +else
> +# Target package, or host package with explicit HOST_FOO_LICENSE_FILES
>   	@$(foreach F,$($(2)_LICENSE_FILES),$(call legal-license-file,$$($(2)_RAWNAME),$(F),$$($(2)_DIR)/$(F))$$(sep))

 Try

	@$$(foreach F,$$($(2)_LICENSE_FILES),$$(call legal-license-file,$$($(2)_RAWNAME),$$(F),$$($(2)_DIR)/$$(F))$$(sep))

 Only the $(2) is expanded immediately, the rest will be expanded when
the legal-info target is executed.

 Regards,
 Arnout

> -endif
> +endif # host/target distinction
> +endif # license files
> +
>   ifeq ($$($(2)_REDISTRIBUTE),YES)
>   # Copy the source tarball (just hardlink if possible)
>   	@cp -l $(DL_DIR)/$$($(2)_SOURCE) $(REDIST_SOURCES_DIR) 2>/dev/null || \
>   	   cp $(DL_DIR)/$$($(2)_SOURCE) $(REDIST_SOURCES_DIR)
> -endif
> -endif
> +endif # redistribute
> +
> +endif # other packages
>   	@$(call legal-manifest,$$($(2)_RAWNAME),$$($(2)_VERSION),$$($(2)_LICENSE),$$($(2)_MANIFEST_LICENSE_FILES),$$($(2)_MANIFEST_TARBALL))
>   endif # ifneq ($(call qstrip,$$($(2)_SOURCE)),)
>   	$(foreach hook,$($(2)_POST_LEGAL_INFO_HOOKS),$(call $(hook))$(sep))
> 
> 


-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

[Buildroot] [PATCH 1 of 6 v2] legal info: fix saving of host package licenses

[Thomas De Schampheleire]

Hi Arnout, all,

On Tue, Nov 12, 2013 at 8:12 AM, Arnout Vandecappelle <arnout@mind.be> wrote:
> On 07/10/13 15:15, Thomas De Schampheleire wrote:
>> Due to some tricky make behavior, the license texts of host packages that
>> did not provide an explicit HOST_FOO_LICENSE_FILES definition was not saved.
>> The problem is that you cannot correctly use a variable defined/updated
>> inside a call'ed block as input to a foreach statement. If you try to use
>> $(FOO) then only the original value of FOO is used for foreach, any update
>> inside the call'ed block is ignored. However, if you use $$(FOO), the entire
>> contents of FOO (typically a list of items) is passed as one item to
>> foreach, thus causing just one iteration instead of several.
>
>  The description of the problem is wrong... It has nothing to do with call,
> it is due to the eval behaviour.
>
>  Any variable referenced with a single $ inside the inner-generic-package macro
> is expanded before the resulting contents are eval'ed. Therefore, it is not
> possible to refer to variables defined by the inner-generic-package macro
> from within a single-$ function call.
>
>  Fortunately there is a simple solution: use $$. See below...

I object to the use of the word 'simple' here!

[..]

>  Try
>
>         @$$(foreach F,$$($(2)_LICENSE_FILES),$$(call legal-license-file,$$($(2)_RAWNAME),$$(F),$$($(2)_DIR)/$$(F))$$(sep))
>
>  Only the $(2) is expanded immediately, the rest will be expanded when
> the legal-info target is executed.

And indeed, it works.
I did try adding double dollar signs in some places, but more randomly
than with insight.

Thanks a lot, I'll send an updated patch!

Best regards,
Thomas