[Buildroot] [PATCH 1/3] Makefile: really generate glibc locales in parallel

[Buildroot] [PATCH 1/3] Makefile: really generate glibc locales in parallel

[yann.morin]

To generate the glibc locale data, we call into a recursive Makefile,
so as to generate locales in parallel. This is done as part of a
target-fialize hook.

However, that hook is registered after all packages have been parsed,
and as such, it maye be registered after hooks defined in packages.

Furthermore, the expansion of target-finalize hooks is done in a recipe,
so it is not easy to understand whether this generates a "simple" rule
or not.

As a consequence, despite the use of $(MAKE), make may not notice that
the command is a recursive call, and will decide to close the jobserver
file-descriptors, yieldiong warnings like:
    make[2]: warning: jobserver unavailable: using -j1.  Add '+' to
    parent make rule.

This causes the lcoale data to not be generated in parallel, which is
initially all the fuss about using a sub-makefile...

So, do as suggested, and prepend the hook with a '+', so that it
isexplicit to make that it should not close its jobserver fds.

Fixes: 6fbdf5159607 (Makefile: Parallelize glibc locale generation)

Signed-off-by: Yann E. MORIN <yann.morin@orange.com>
Cc: Gleb Mazovetskiy <glex.spb@gmail.com>
---
 Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Makefile b/Makefile
index ec7c034ac1..ededfa491d 100644
--- a/Makefile
+++ b/Makefile
@@ -653,7 +653,7 @@ ifneq ($(GLIBC_GENERATE_LOCALES),)
 PACKAGES += host-localedef
 
 define GENERATE_GLIBC_LOCALES
-	$(MAKE) -f support/misc/gen-glibc-locales.mk \
+	+$(MAKE) -f support/misc/gen-glibc-locales.mk \
 		ENDIAN=$(call LOWERCASE,$(BR2_ENDIAN)) \
 		LOCALES="$(GLIBC_GENERATE_LOCALES)" \
 		Q=$(Q)
-- 
2.25.1


_________________________________________________________________________________________________________________________

Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

[Buildroot] [PATCH 2/3] support/scripts: don't require gawk to generate glibc gconv modules

[yann.morin]

When a subset of the glibc gconv modules installed, we eed to generate a
trimmed-down list of available modules. We currently use gawk for that.

However, we are not using any GNU extension in that awk script, and it
happens to work as expected when using mawk (which has no GNU
extension).

Commit 11c1076db9a5 (toolchain: add option to copy the gconv libraries)
did not explain why it used gawk explicitly, and given the age for that
commit, we doubt we'd be able to have the involved participants recall
anything from that period...

Besides, gawk is not a requirement for Buildroot.

Switch over to using plain awk.

Signed-off-by: Yann E. MORIN <yann.morin@orange.com>
---
 support/scripts/expunge-gconv-modules | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/support/scripts/expunge-gconv-modules b/support/scripts/expunge-gconv-modules
index 03012c1ce3..bc60fc0ce4 100755
--- a/support/scripts/expunge-gconv-modules
+++ b/support/scripts/expunge-gconv-modules
@@ -19,7 +19,7 @@
 # we handle each with slightly different code, since the second never has
 # associated aliases.
 
-gawk -v files="${1}" '
+awk -v files="${2}" '
 $1 == "alias" {
     aliases[$3] = aliases[$3] " " $2;
 }
-- 
2.25.1


_________________________________________________________________________________________________________________________

Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

[Buildroot] [PATCH 3/3] toolchain: suppot gconv modules fro mglibc >= 2.34

[yann.morin]

Startig with glibc 2.34, the gconv modules description has been split in
two:
  - a common definition in the old location, /usr/lib/gconv/gconv-modules
  - specific defitions in a subdirectory, /usr/lib/gconv/gconv-modules.d/

This is done so as to simplify the handling of glibc gconv modules, and
eventually to segregate those outside of glibc, and so that third-parties
may also provide their own gconv converters and their definitions.

And starting with that same glibc version, most of the gconv modules
definition is moved to an extra configuration file in that
sub-directory.

It is thus no longer possible to use special code pages, like cp850,
which are very usefull to access FAT-formatted devices.

Add suppot for this new gconv layout, while keeping support for older
glibc versions. Note that the modules themselves are not moved or
renaed, just the definition files have changed.

Instead of passing the one old gonv modules definitions file on stdin,
we pass the base directory to that file, and move into the script the
responsibility to find all the gconv definition files.

Signed-off-by: Yann E. MORIN <yann.morin@orange.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Romain Naour <romain.naour@gmail.com>
Cc: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Cc: Giulio Benetti <giulio.benetti@benettiengineering.com>
---
 support/scripts/expunge-gconv-modules | 22 ++++++++++++++++------
 toolchain/toolchain.mk                |  9 +++++++--
 2 files changed, 23 insertions(+), 8 deletions(-)

diff --git a/support/scripts/expunge-gconv-modules b/support/scripts/expunge-gconv-modules
index bc60fc0ce4..e9ac48ca3f 100755
--- a/support/scripts/expunge-gconv-modules
+++ b/support/scripts/expunge-gconv-modules
@@ -1,11 +1,17 @@
 #!/usr/bin/env bash
 
 # This script is used to generate a gconv-modules file that takes into
-# account only the gconv modules installed by Buildroot. It receives
-# on its standard input the original complete gconv-modules file from
-# the toolchain, and as arguments the list of gconv modules that were
-# actually installed, and writes on its standard output the new
-# gconv-modules file.
+# account only the gconv modules installed by Buildroot, and generates
+# a stripped-down gconv-moduels file on its stdout.
+# It takes two arguments:
+#   $1: the directory where to look for gconv modules definitions
+#   $2: a space-separated list of gconv modules that were actually
+#       installed
+
+# Starting with glibc-2.34, modules definitions are located in multiple
+# files:
+#   ${1}/gconv-modules
+#   ${1}/gconv-modules.d/*.conf
 
 # The format of gconv-modules is precisely documented in the
 # file itself. It consists of two different directives:
@@ -19,7 +25,11 @@
 # we handle each with slightly different code, since the second never has
 # associated aliases.
 
-awk -v files="${2}" '
+for f in ${1}/gconv-modules ${1}/gconv-modules.d/*.conf; do
+    [ -e "${f}" ] || continue
+    cat "${f}"
+done \
+|awk -v files="${2}" '
 $1 == "alias" {
     aliases[$3] = aliases[$3] " " $2;
 }
diff --git a/toolchain/toolchain.mk b/toolchain/toolchain.mk
index 08d1649603..fe87a72ed4 100644
--- a/toolchain/toolchain.mk
+++ b/toolchain/toolchain.mk
@@ -27,6 +27,10 @@ define TOOLCHAIN_GLIBC_COPY_GCONV_LIBS
 		$(INSTALL) -m 0644 $(STAGING_DIR)/usr/lib/$${d}/gconv/*.so \
 				   $(TARGET_DIR)/usr/lib/gconv \
 		|| exit 1; \
+		if [ -d $(STAGING_DIR)/usr/lib/$${d}/gconv/gconv-modules.d ]; then \
+			cp -a $(STAGING_DIR)/usr/lib/$${d}/gconv/gconv-modules.d \
+				$(TARGET_DIR)/usr/lib/gconv/ || exit 1; \
+		fi; \
 	else \
 		for l in $(TOOLCHAIN_GLIBC_GCONV_LIBS); do \
 			$(INSTALL) -m 0644 -D $(STAGING_DIR)/usr/lib/$${d}/gconv/$${l}.so \
@@ -41,8 +45,9 @@ define TOOLCHAIN_GLIBC_COPY_GCONV_LIBS
 				 || exit 1; \
 			done; \
 		done; \
-		./support/scripts/expunge-gconv-modules "$(TOOLCHAIN_GLIBC_GCONV_LIBS)" \
-			<$(STAGING_DIR)/usr/lib/$${d}/gconv/gconv-modules \
+		./support/scripts/expunge-gconv-modules \
+			$(STAGING_DIR)/usr/lib/$${d}/gconv \
+			"$(TOOLCHAIN_GLIBC_GCONV_LIBS)" \
 			>$(TARGET_DIR)/usr/lib/gconv/gconv-modules; \
 	fi
 endef
-- 
2.25.1


_________________________________________________________________________________________________________________________

Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 1/3] Makefile: really generate glibc locales in parallel

[Yann E. MORIN]

Yann, All,

On 2022-10-14 13:18 +0200, yann.morin@orange.com spake thusly:
> To generate the glibc locale data, we call into a recursive Makefile,
> so as to generate locales in parallel. This is done as part of a
> target-fialize hook.

*finalize

> However, that hook is registered after all packages have been parsed,
> and as such, it maye be registered after hooks defined in packages.
> 
> Furthermore, the expansion of target-finalize hooks is done in a recipe,
> so it is not easy to understand whether this generates a "simple" rule
> or not.
> 
> As a consequence, despite the use of $(MAKE), make may not notice that
> the command is a recursive call, and will decide to close the jobserver
> file-descriptors, yieldiong warnings like:

*yielding

>     make[2]: warning: jobserver unavailable: using -j1.  Add '+' to
>     parent make rule.
> 
> This causes the lcoale data to not be generated in parallel, which is

*locale (damn, I forgot to fix that one when applying)

> initially all the fuss about using a sub-makefile...
> 
> So, do as suggested, and prepend the hook with a '+', so that it
> isexplicit to make that it should not close its jobserver fds.

*is explicit

> Fixes: 6fbdf5159607 (Makefile: Parallelize glibc locale generation)
> 
> Signed-off-by: Yann E. MORIN <yann.morin@orange.com>
> Cc: Gleb Mazovetskiy <glex.spb@gmail.com>

Applied to master with the typoes fixed, thanks.

Regards,
Yann E. MORIN.

> ---
>  Makefile | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/Makefile b/Makefile
> index ec7c034ac1..ededfa491d 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -653,7 +653,7 @@ ifneq ($(GLIBC_GENERATE_LOCALES),)
>  PACKAGES += host-localedef
>  
>  define GENERATE_GLIBC_LOCALES
> -	$(MAKE) -f support/misc/gen-glibc-locales.mk \
> +	+$(MAKE) -f support/misc/gen-glibc-locales.mk \
>  		ENDIAN=$(call LOWERCASE,$(BR2_ENDIAN)) \
>  		LOCALES="$(GLIBC_GENERATE_LOCALES)" \
>  		Q=$(Q)
> -- 
> 2.25.1
> 
> 
> _________________________________________________________________________________________________________________________
> 
> Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
> pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
> a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
> Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.
> 
> This message and its attachments may contain confidential or privileged information that may be protected by law;
> they should not be distributed, used or copied without authorisation.
> If you have received this email in error, please notify the sender and delete this message and its attachments.
> As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
> Thank you.
> 
> _______________________________________________
> buildroot mailing list
> buildroot@buildroot.org
> https://lists.buildroot.org/mailman/listinfo/buildroot

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 2/3] support/scripts: don't require gawk to generate glibc gconv modules

[Yann E. MORIN]

Yann, All,

On 2022-10-14 13:18 +0200, yann.morin@orange.com spake thusly:
> When a subset of the glibc gconv modules installed, we eed to generate a

*are installed, we need

> trimmed-down list of available modules. We currently use gawk for that.
> 
> However, we are not using any GNU extension in that awk script, and it
> happens to work as expected when using mawk (which has no GNU
> extension).
> 
> Commit 11c1076db9a5 (toolchain: add option to copy the gconv libraries)
> did not explain why it used gawk explicitly, and given the age for that
> commit, we doubt we'd be able to have the involved participants recall
> anything from that period...
> 
> Besides, gawk is not a requirement for Buildroot.
> 
> Switch over to using plain awk.
> 
> Signed-off-by: Yann E. MORIN <yann.morin@orange.com>

Applied to master with the typoes fixed, thanks.

Regards,
Yann E. MORIN.

> ---
>  support/scripts/expunge-gconv-modules | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/support/scripts/expunge-gconv-modules b/support/scripts/expunge-gconv-modules
> index 03012c1ce3..bc60fc0ce4 100755
> --- a/support/scripts/expunge-gconv-modules
> +++ b/support/scripts/expunge-gconv-modules
> @@ -19,7 +19,7 @@
>  # we handle each with slightly different code, since the second never has
>  # associated aliases.
>  
> -gawk -v files="${1}" '
> +awk -v files="${2}" '
>  $1 == "alias" {
>      aliases[$3] = aliases[$3] " " $2;
>  }
> -- 
> 2.25.1
> 
> 
> _________________________________________________________________________________________________________________________
> 
> Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
> pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
> a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
> Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.
> 
> This message and its attachments may contain confidential or privileged information that may be protected by law;
> they should not be distributed, used or copied without authorisation.
> If you have received this email in error, please notify the sender and delete this message and its attachments.
> As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
> Thank you.
> 
> _______________________________________________
> buildroot mailing list
> buildroot@buildroot.org
> https://lists.buildroot.org/mailman/listinfo/buildroot

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 3/3] toolchain: suppot gconv modules fro mglibc >= 2.34

[Yann E. MORIN]

Yann, All,

In the title: *support, from glibc

Also, many typoes in the commit log...

On 2022-10-14 13:18 +0200, yann.morin@orange.com spake thusly:
> Startig with glibc 2.34, the gconv modules description has been split in
> two:
>   - a common definition in the old location, /usr/lib/gconv/gconv-modules
>   - specific defitions in a subdirectory, /usr/lib/gconv/gconv-modules.d/
> 
> This is done so as to simplify the handling of glibc gconv modules, and
> eventually to segregate those outside of glibc, and so that third-parties
> may also provide their own gconv converters and their definitions.
> 
> And starting with that same glibc version, most of the gconv modules
> definition is moved to an extra configuration file in that
> sub-directory.
> 
> It is thus no longer possible to use special code pages, like cp850,
> which are very usefull to access FAT-formatted devices.
> 
> Add suppot for this new gconv layout, while keeping support for older
> glibc versions. Note that the modules themselves are not moved or
> renaed, just the definition files have changed.
> 
> Instead of passing the one old gonv modules definitions file on stdin,
> we pass the base directory to that file, and move into the script the
> responsibility to find all the gconv definition files.
> 
> Signed-off-by: Yann E. MORIN <yann.morin@orange.com>
> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
> Cc: Romain Naour <romain.naour@gmail.com>
> Cc: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
> Cc: Giulio Benetti <giulio.benetti@benettiengineering.com>

Applied to master with the many typoes fixed, thanks.

Regards,
Yann E. MORIN.

> ---
>  support/scripts/expunge-gconv-modules | 22 ++++++++++++++++------
>  toolchain/toolchain.mk                |  9 +++++++--
>  2 files changed, 23 insertions(+), 8 deletions(-)
> 
> diff --git a/support/scripts/expunge-gconv-modules b/support/scripts/expunge-gconv-modules
> index bc60fc0ce4..e9ac48ca3f 100755
> --- a/support/scripts/expunge-gconv-modules
> +++ b/support/scripts/expunge-gconv-modules
> @@ -1,11 +1,17 @@
>  #!/usr/bin/env bash
>  
>  # This script is used to generate a gconv-modules file that takes into
> -# account only the gconv modules installed by Buildroot. It receives
> -# on its standard input the original complete gconv-modules file from
> -# the toolchain, and as arguments the list of gconv modules that were
> -# actually installed, and writes on its standard output the new
> -# gconv-modules file.
> +# account only the gconv modules installed by Buildroot, and generates
> +# a stripped-down gconv-moduels file on its stdout.
> +# It takes two arguments:
> +#   $1: the directory where to look for gconv modules definitions
> +#   $2: a space-separated list of gconv modules that were actually
> +#       installed
> +
> +# Starting with glibc-2.34, modules definitions are located in multiple
> +# files:
> +#   ${1}/gconv-modules
> +#   ${1}/gconv-modules.d/*.conf
>  
>  # The format of gconv-modules is precisely documented in the
>  # file itself. It consists of two different directives:
> @@ -19,7 +25,11 @@
>  # we handle each with slightly different code, since the second never has
>  # associated aliases.
>  
> -awk -v files="${2}" '
> +for f in ${1}/gconv-modules ${1}/gconv-modules.d/*.conf; do
> +    [ -e "${f}" ] || continue
> +    cat "${f}"
> +done \
> +|awk -v files="${2}" '
>  $1 == "alias" {
>      aliases[$3] = aliases[$3] " " $2;
>  }
> diff --git a/toolchain/toolchain.mk b/toolchain/toolchain.mk
> index 08d1649603..fe87a72ed4 100644
> --- a/toolchain/toolchain.mk
> +++ b/toolchain/toolchain.mk
> @@ -27,6 +27,10 @@ define TOOLCHAIN_GLIBC_COPY_GCONV_LIBS
>  		$(INSTALL) -m 0644 $(STAGING_DIR)/usr/lib/$${d}/gconv/*.so \
>  				   $(TARGET_DIR)/usr/lib/gconv \
>  		|| exit 1; \
> +		if [ -d $(STAGING_DIR)/usr/lib/$${d}/gconv/gconv-modules.d ]; then \
> +			cp -a $(STAGING_DIR)/usr/lib/$${d}/gconv/gconv-modules.d \
> +				$(TARGET_DIR)/usr/lib/gconv/ || exit 1; \
> +		fi; \
>  	else \
>  		for l in $(TOOLCHAIN_GLIBC_GCONV_LIBS); do \
>  			$(INSTALL) -m 0644 -D $(STAGING_DIR)/usr/lib/$${d}/gconv/$${l}.so \
> @@ -41,8 +45,9 @@ define TOOLCHAIN_GLIBC_COPY_GCONV_LIBS
>  				 || exit 1; \
>  			done; \
>  		done; \
> -		./support/scripts/expunge-gconv-modules "$(TOOLCHAIN_GLIBC_GCONV_LIBS)" \
> -			<$(STAGING_DIR)/usr/lib/$${d}/gconv/gconv-modules \
> +		./support/scripts/expunge-gconv-modules \
> +			$(STAGING_DIR)/usr/lib/$${d}/gconv \
> +			"$(TOOLCHAIN_GLIBC_GCONV_LIBS)" \
>  			>$(TARGET_DIR)/usr/lib/gconv/gconv-modules; \
>  	fi
>  endef
> -- 
> 2.25.1
> 
> 
> _________________________________________________________________________________________________________________________
> 
> Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
> pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
> a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
> Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.
> 
> This message and its attachments may contain confidential or privileged information that may be protected by law;
> they should not be distributed, used or copied without authorisation.
> If you have received this email in error, please notify the sender and delete this message and its attachments.
> As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
> Thank you.
> 
> _______________________________________________
> buildroot mailing list
> buildroot@buildroot.org
> https://lists.buildroot.org/mailman/listinfo/buildroot

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 1/3] Makefile: really generate glibc locales in parallel

[Yann E. MORIN]

Yann, All,

Your git send-email setup is incorrect: the From field is missing the
realname part (which is weird, as it was not missing in the previous
gpsd patches)...

Regards,
Yann E. MORIN.

On 2022-10-14 13:18 +0200, yann.morin@orange.com spake thusly:
> To generate the glibc locale data, we call into a recursive Makefile,
> so as to generate locales in parallel. This is done as part of a
> target-fialize hook.
> 
> However, that hook is registered after all packages have been parsed,
> and as such, it maye be registered after hooks defined in packages.
> 
> Furthermore, the expansion of target-finalize hooks is done in a recipe,
> so it is not easy to understand whether this generates a "simple" rule
> or not.
> 
> As a consequence, despite the use of $(MAKE), make may not notice that
> the command is a recursive call, and will decide to close the jobserver
> file-descriptors, yieldiong warnings like:
>     make[2]: warning: jobserver unavailable: using -j1.  Add '+' to
>     parent make rule.
> 
> This causes the lcoale data to not be generated in parallel, which is
> initially all the fuss about using a sub-makefile...
> 
> So, do as suggested, and prepend the hook with a '+', so that it
> isexplicit to make that it should not close its jobserver fds.
> 
> Fixes: 6fbdf5159607 (Makefile: Parallelize glibc locale generation)
> 
> Signed-off-by: Yann E. MORIN <yann.morin@orange.com>
> Cc: Gleb Mazovetskiy <glex.spb@gmail.com>
> ---
>  Makefile | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/Makefile b/Makefile
> index ec7c034ac1..ededfa491d 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -653,7 +653,7 @@ ifneq ($(GLIBC_GENERATE_LOCALES),)
>  PACKAGES += host-localedef
>  
>  define GENERATE_GLIBC_LOCALES
> -	$(MAKE) -f support/misc/gen-glibc-locales.mk \
> +	+$(MAKE) -f support/misc/gen-glibc-locales.mk \
>  		ENDIAN=$(call LOWERCASE,$(BR2_ENDIAN)) \
>  		LOCALES="$(GLIBC_GENERATE_LOCALES)" \
>  		Q=$(Q)
> -- 
> 2.25.1
> 
> 
> _________________________________________________________________________________________________________________________
> 
> Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
> pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
> a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
> Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.
> 
> This message and its attachments may contain confidential or privileged information that may be protected by law;
> they should not be distributed, used or copied without authorisation.
> If you have received this email in error, please notify the sender and delete this message and its attachments.
> As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
> Thank you.
> 
> _______________________________________________
> buildroot mailing list
> buildroot@buildroot.org
> https://lists.buildroot.org/mailman/listinfo/buildroot

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 1/3] Makefile: really generate glibc locales in parallel

[Peter Korsgaard]

>>>>>   <yann.morin@orange.com> writes:

 > To generate the glibc locale data, we call into a recursive Makefile,
 > so as to generate locales in parallel. This is done as part of a
 > target-fialize hook.

 > However, that hook is registered after all packages have been parsed,
 > and as such, it maye be registered after hooks defined in packages.

 > Furthermore, the expansion of target-finalize hooks is done in a recipe,
 > so it is not easy to understand whether this generates a "simple" rule
 > or not.

 > As a consequence, despite the use of $(MAKE), make may not notice that
 > the command is a recursive call, and will decide to close the jobserver
 > file-descriptors, yieldiong warnings like:
 >     make[2]: warning: jobserver unavailable: using -j1.  Add '+' to
 >     parent make rule.

 > This causes the lcoale data to not be generated in parallel, which is
 > initially all the fuss about using a sub-makefile...

 > So, do as suggested, and prepend the hook with a '+', so that it
 > isexplicit to make that it should not close its jobserver fds.

 > Fixes: 6fbdf5159607 (Makefile: Parallelize glibc locale generation)

 > Signed-off-by: Yann E. MORIN <yann.morin@orange.com>
 > Cc: Gleb Mazovetskiy <glex.spb@gmail.com>

Committed to 2022.08.x and 2022.02.x, thanks.

-- 
Bye, Peter Korsgaard
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 2/3] support/scripts: don't require gawk to generate glibc gconv modules

[Peter Korsgaard]

>>>>>   <yann.morin@orange.com> writes:

 > When a subset of the glibc gconv modules installed, we eed to generate a
 > trimmed-down list of available modules. We currently use gawk for that.

 > However, we are not using any GNU extension in that awk script, and it
 > happens to work as expected when using mawk (which has no GNU
 > extension).

 > Commit 11c1076db9a5 (toolchain: add option to copy the gconv libraries)
 > did not explain why it used gawk explicitly, and given the age for that
 > commit, we doubt we'd be able to have the involved participants recall
 > anything from that period...

 > Besides, gawk is not a requirement for Buildroot.

 > Switch over to using plain awk.

 > Signed-off-by: Yann E. MORIN <yann.morin@orange.com>

Committed to 2022.08.x and 2022.02.x, thanks.

-- 
Bye, Peter Korsgaard
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 3/3] toolchain: suppot gconv modules fro mglibc >= 2.34

[Peter Korsgaard]

>>>>>   <yann.morin@orange.com> writes:

 > Startig with glibc 2.34, the gconv modules description has been split in
 > two:
 >   - a common definition in the old location, /usr/lib/gconv/gconv-modules
 >   - specific defitions in a subdirectory, /usr/lib/gconv/gconv-modules.d/

 > This is done so as to simplify the handling of glibc gconv modules, and
 > eventually to segregate those outside of glibc, and so that third-parties
 > may also provide their own gconv converters and their definitions.

 > And starting with that same glibc version, most of the gconv modules
 > definition is moved to an extra configuration file in that
 > sub-directory.

 > It is thus no longer possible to use special code pages, like cp850,
 > which are very usefull to access FAT-formatted devices.

 > Add suppot for this new gconv layout, while keeping support for older
 > glibc versions. Note that the modules themselves are not moved or
 > renaed, just the definition files have changed.

 > Instead of passing the one old gonv modules definitions file on stdin,
 > we pass the base directory to that file, and move into the script the
 > responsibility to find all the gconv definition files.

 > Signed-off-by: Yann E. MORIN <yann.morin@orange.com>
 > Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
 > Cc: Romain Naour <romain.naour@gmail.com>
 > Cc: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
 > Cc: Giulio Benetti <giulio.benetti@benettiengineering.com>

Committed to 2022.08.x and 2022.02.x, thanks.

-- 
Bye, Peter Korsgaard
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot