* [Buildroot] [PATCH 1/3] Makefile: really generate glibc locales in parallel @ 2022-10-14 11:18 yann.morin 2022-10-21 19:07 ` Yann E. MORIN ` (2 more replies) 0 siblings, 3 replies; 10+ messages in thread From: yann.morin @ 2022-10-14 11:18 UTC (permalink / raw) To: buildroot; +Cc: Gleb Mazovetskiy, yann.morin To generate the glibc locale data, we call into a recursive Makefile, so as to generate locales in parallel. This is done as part of a target-fialize hook. However, that hook is registered after all packages have been parsed, and as such, it maye be registered after hooks defined in packages. Furthermore, the expansion of target-finalize hooks is done in a recipe, so it is not easy to understand whether this generates a "simple" rule or not. As a consequence, despite the use of $(MAKE), make may not notice that the command is a recursive call, and will decide to close the jobserver file-descriptors, yieldiong warnings like: make[2]: warning: jobserver unavailable: using -j1. Add '+' to parent make rule. This causes the lcoale data to not be generated in parallel, which is initially all the fuss about using a sub-makefile... So, do as suggested, and prepend the hook with a '+', so that it isexplicit to make that it should not close its jobserver fds. Fixes: 6fbdf5159607 (Makefile: Parallelize glibc locale generation) Signed-off-by: Yann E. MORIN <yann.morin@orange.com> Cc: Gleb Mazovetskiy <glex.spb@gmail.com> --- Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Makefile b/Makefile index ec7c034ac1..ededfa491d 100644 --- a/Makefile +++ b/Makefile @@ -653,7 +653,7 @@ ifneq ($(GLIBC_GENERATE_LOCALES),) PACKAGES += host-localedef define GENERATE_GLIBC_LOCALES - $(MAKE) -f support/misc/gen-glibc-locales.mk \ + +$(MAKE) -f support/misc/gen-glibc-locales.mk \ ENDIAN=$(call LOWERCASE,$(BR2_ENDIAN)) \ LOCALES="$(GLIBC_GENERATE_LOCALES)" \ Q=$(Q) -- 2.25.1 _________________________________________________________________________________________________________________________ Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. If you have received this email in error, please notify the sender and delete this message and its attachments. As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified. Thank you. _______________________________________________ buildroot mailing list buildroot@buildroot.org https://lists.buildroot.org/mailman/listinfo/buildroot ^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: [Buildroot] [PATCH 1/3] Makefile: really generate glibc locales in parallel 2022-10-14 11:18 [Buildroot] [PATCH 1/3] Makefile: really generate glibc locales in parallel yann.morin @ 2022-10-21 19:07 ` Yann E. MORIN 2022-10-21 19:46 ` Yann E. MORIN 2022-11-03 13:49 ` Peter Korsgaard 2 siblings, 0 replies; 10+ messages in thread From: Yann E. MORIN @ 2022-10-21 19:07 UTC (permalink / raw) To: yann.morin; +Cc: Gleb Mazovetskiy, buildroot Yann, All, On 2022-10-14 13:18 +0200, yann.morin@orange.com spake thusly: > To generate the glibc locale data, we call into a recursive Makefile, > so as to generate locales in parallel. This is done as part of a > target-fialize hook. *finalize > However, that hook is registered after all packages have been parsed, > and as such, it maye be registered after hooks defined in packages. > > Furthermore, the expansion of target-finalize hooks is done in a recipe, > so it is not easy to understand whether this generates a "simple" rule > or not. > > As a consequence, despite the use of $(MAKE), make may not notice that > the command is a recursive call, and will decide to close the jobserver > file-descriptors, yieldiong warnings like: *yielding > make[2]: warning: jobserver unavailable: using -j1. Add '+' to > parent make rule. > > This causes the lcoale data to not be generated in parallel, which is *locale (damn, I forgot to fix that one when applying) > initially all the fuss about using a sub-makefile... > > So, do as suggested, and prepend the hook with a '+', so that it > isexplicit to make that it should not close its jobserver fds. *is explicit > Fixes: 6fbdf5159607 (Makefile: Parallelize glibc locale generation) > > Signed-off-by: Yann E. MORIN <yann.morin@orange.com> > Cc: Gleb Mazovetskiy <glex.spb@gmail.com> Applied to master with the typoes fixed, thanks. Regards, Yann E. MORIN. > --- > Makefile | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/Makefile b/Makefile > index ec7c034ac1..ededfa491d 100644 > --- a/Makefile > +++ b/Makefile > @@ -653,7 +653,7 @@ ifneq ($(GLIBC_GENERATE_LOCALES),) > PACKAGES += host-localedef > > define GENERATE_GLIBC_LOCALES > - $(MAKE) -f support/misc/gen-glibc-locales.mk \ > + +$(MAKE) -f support/misc/gen-glibc-locales.mk \ > ENDIAN=$(call LOWERCASE,$(BR2_ENDIAN)) \ > LOCALES="$(GLIBC_GENERATE_LOCALES)" \ > Q=$(Q) > -- > 2.25.1 > > > _________________________________________________________________________________________________________________________ > > Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc > pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler > a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, > Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. > > This message and its attachments may contain confidential or privileged information that may be protected by law; > they should not be distributed, used or copied without authorisation. > If you have received this email in error, please notify the sender and delete this message and its attachments. > As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified. > Thank you. > > _______________________________________________ > buildroot mailing list > buildroot@buildroot.org > https://lists.buildroot.org/mailman/listinfo/buildroot -- .-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 561 099 427 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------' _______________________________________________ buildroot mailing list buildroot@buildroot.org https://lists.buildroot.org/mailman/listinfo/buildroot ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [Buildroot] [PATCH 1/3] Makefile: really generate glibc locales in parallel 2022-10-14 11:18 [Buildroot] [PATCH 1/3] Makefile: really generate glibc locales in parallel yann.morin 2022-10-21 19:07 ` Yann E. MORIN @ 2022-10-21 19:46 ` Yann E. MORIN 2022-11-03 13:49 ` Peter Korsgaard 2 siblings, 0 replies; 10+ messages in thread From: Yann E. MORIN @ 2022-10-21 19:46 UTC (permalink / raw) To: yann.morin; +Cc: Gleb Mazovetskiy, buildroot Yann, All, Your git send-email setup is incorrect: the From field is missing the realname part (which is weird, as it was not missing in the previous gpsd patches)... Regards, Yann E. MORIN. On 2022-10-14 13:18 +0200, yann.morin@orange.com spake thusly: > To generate the glibc locale data, we call into a recursive Makefile, > so as to generate locales in parallel. This is done as part of a > target-fialize hook. > > However, that hook is registered after all packages have been parsed, > and as such, it maye be registered after hooks defined in packages. > > Furthermore, the expansion of target-finalize hooks is done in a recipe, > so it is not easy to understand whether this generates a "simple" rule > or not. > > As a consequence, despite the use of $(MAKE), make may not notice that > the command is a recursive call, and will decide to close the jobserver > file-descriptors, yieldiong warnings like: > make[2]: warning: jobserver unavailable: using -j1. Add '+' to > parent make rule. > > This causes the lcoale data to not be generated in parallel, which is > initially all the fuss about using a sub-makefile... > > So, do as suggested, and prepend the hook with a '+', so that it > isexplicit to make that it should not close its jobserver fds. > > Fixes: 6fbdf5159607 (Makefile: Parallelize glibc locale generation) > > Signed-off-by: Yann E. MORIN <yann.morin@orange.com> > Cc: Gleb Mazovetskiy <glex.spb@gmail.com> > --- > Makefile | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/Makefile b/Makefile > index ec7c034ac1..ededfa491d 100644 > --- a/Makefile > +++ b/Makefile > @@ -653,7 +653,7 @@ ifneq ($(GLIBC_GENERATE_LOCALES),) > PACKAGES += host-localedef > > define GENERATE_GLIBC_LOCALES > - $(MAKE) -f support/misc/gen-glibc-locales.mk \ > + +$(MAKE) -f support/misc/gen-glibc-locales.mk \ > ENDIAN=$(call LOWERCASE,$(BR2_ENDIAN)) \ > LOCALES="$(GLIBC_GENERATE_LOCALES)" \ > Q=$(Q) > -- > 2.25.1 > > > _________________________________________________________________________________________________________________________ > > Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc > pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler > a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, > Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. > > This message and its attachments may contain confidential or privileged information that may be protected by law; > they should not be distributed, used or copied without authorisation. > If you have received this email in error, please notify the sender and delete this message and its attachments. > As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified. > Thank you. > > _______________________________________________ > buildroot mailing list > buildroot@buildroot.org > https://lists.buildroot.org/mailman/listinfo/buildroot -- .-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 561 099 427 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------' _______________________________________________ buildroot mailing list buildroot@buildroot.org https://lists.buildroot.org/mailman/listinfo/buildroot ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [Buildroot] [PATCH 1/3] Makefile: really generate glibc locales in parallel 2022-10-14 11:18 [Buildroot] [PATCH 1/3] Makefile: really generate glibc locales in parallel yann.morin 2022-10-21 19:07 ` Yann E. MORIN 2022-10-21 19:46 ` Yann E. MORIN @ 2022-11-03 13:49 ` Peter Korsgaard 2 siblings, 0 replies; 10+ messages in thread From: Peter Korsgaard @ 2022-11-03 13:49 UTC (permalink / raw) To: yann.morin; +Cc: Gleb Mazovetskiy, buildroot >>>>> <yann.morin@orange.com> writes: > To generate the glibc locale data, we call into a recursive Makefile, > so as to generate locales in parallel. This is done as part of a > target-fialize hook. > However, that hook is registered after all packages have been parsed, > and as such, it maye be registered after hooks defined in packages. > Furthermore, the expansion of target-finalize hooks is done in a recipe, > so it is not easy to understand whether this generates a "simple" rule > or not. > As a consequence, despite the use of $(MAKE), make may not notice that > the command is a recursive call, and will decide to close the jobserver > file-descriptors, yieldiong warnings like: > make[2]: warning: jobserver unavailable: using -j1. Add '+' to > parent make rule. > This causes the lcoale data to not be generated in parallel, which is > initially all the fuss about using a sub-makefile... > So, do as suggested, and prepend the hook with a '+', so that it > isexplicit to make that it should not close its jobserver fds. > Fixes: 6fbdf5159607 (Makefile: Parallelize glibc locale generation) > Signed-off-by: Yann E. MORIN <yann.morin@orange.com> > Cc: Gleb Mazovetskiy <glex.spb@gmail.com> Committed to 2022.08.x and 2022.02.x, thanks. -- Bye, Peter Korsgaard _______________________________________________ buildroot mailing list buildroot@buildroot.org https://lists.buildroot.org/mailman/listinfo/buildroot ^ permalink raw reply [flat|nested] 10+ messages in thread
[parent not found: <2b0b6b796ff36967c963ba90f138a03d467ee5cf.1665746266.git.yann.morin@orange.com>]
* [Buildroot] [PATCH 2/3] support/scripts: don't require gawk to generate glibc gconv modules [not found] <2b0b6b796ff36967c963ba90f138a03d467ee5cf.1665746266.git.yann.morin@orange.com> @ 2022-10-14 11:18 ` yann.morin 2022-10-21 19:08 ` Yann E. MORIN 2022-11-04 7:40 ` Peter Korsgaard 2022-10-14 11:18 ` [Buildroot] [PATCH 3/3] toolchain: suppot gconv modules fro mglibc >= 2.34 yann.morin 1 sibling, 2 replies; 10+ messages in thread From: yann.morin @ 2022-10-14 11:18 UTC (permalink / raw) To: buildroot; +Cc: yann.morin When a subset of the glibc gconv modules installed, we eed to generate a trimmed-down list of available modules. We currently use gawk for that. However, we are not using any GNU extension in that awk script, and it happens to work as expected when using mawk (which has no GNU extension). Commit 11c1076db9a5 (toolchain: add option to copy the gconv libraries) did not explain why it used gawk explicitly, and given the age for that commit, we doubt we'd be able to have the involved participants recall anything from that period... Besides, gawk is not a requirement for Buildroot. Switch over to using plain awk. Signed-off-by: Yann E. MORIN <yann.morin@orange.com> --- support/scripts/expunge-gconv-modules | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/support/scripts/expunge-gconv-modules b/support/scripts/expunge-gconv-modules index 03012c1ce3..bc60fc0ce4 100755 --- a/support/scripts/expunge-gconv-modules +++ b/support/scripts/expunge-gconv-modules @@ -19,7 +19,7 @@ # we handle each with slightly different code, since the second never has # associated aliases. -gawk -v files="${1}" ' +awk -v files="${2}" ' $1 == "alias" { aliases[$3] = aliases[$3] " " $2; } -- 2.25.1 _________________________________________________________________________________________________________________________ Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. If you have received this email in error, please notify the sender and delete this message and its attachments. As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified. Thank you. _______________________________________________ buildroot mailing list buildroot@buildroot.org https://lists.buildroot.org/mailman/listinfo/buildroot ^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: [Buildroot] [PATCH 2/3] support/scripts: don't require gawk to generate glibc gconv modules 2022-10-14 11:18 ` [Buildroot] [PATCH 2/3] support/scripts: don't require gawk to generate glibc gconv modules yann.morin @ 2022-10-21 19:08 ` Yann E. MORIN 2022-11-04 7:40 ` Peter Korsgaard 1 sibling, 0 replies; 10+ messages in thread From: Yann E. MORIN @ 2022-10-21 19:08 UTC (permalink / raw) To: yann.morin; +Cc: buildroot Yann, All, On 2022-10-14 13:18 +0200, yann.morin@orange.com spake thusly: > When a subset of the glibc gconv modules installed, we eed to generate a *are installed, we need > trimmed-down list of available modules. We currently use gawk for that. > > However, we are not using any GNU extension in that awk script, and it > happens to work as expected when using mawk (which has no GNU > extension). > > Commit 11c1076db9a5 (toolchain: add option to copy the gconv libraries) > did not explain why it used gawk explicitly, and given the age for that > commit, we doubt we'd be able to have the involved participants recall > anything from that period... > > Besides, gawk is not a requirement for Buildroot. > > Switch over to using plain awk. > > Signed-off-by: Yann E. MORIN <yann.morin@orange.com> Applied to master with the typoes fixed, thanks. Regards, Yann E. MORIN. > --- > support/scripts/expunge-gconv-modules | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/support/scripts/expunge-gconv-modules b/support/scripts/expunge-gconv-modules > index 03012c1ce3..bc60fc0ce4 100755 > --- a/support/scripts/expunge-gconv-modules > +++ b/support/scripts/expunge-gconv-modules > @@ -19,7 +19,7 @@ > # we handle each with slightly different code, since the second never has > # associated aliases. > > -gawk -v files="${1}" ' > +awk -v files="${2}" ' > $1 == "alias" { > aliases[$3] = aliases[$3] " " $2; > } > -- > 2.25.1 > > > _________________________________________________________________________________________________________________________ > > Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc > pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler > a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, > Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. > > This message and its attachments may contain confidential or privileged information that may be protected by law; > they should not be distributed, used or copied without authorisation. > If you have received this email in error, please notify the sender and delete this message and its attachments. > As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified. > Thank you. > > _______________________________________________ > buildroot mailing list > buildroot@buildroot.org > https://lists.buildroot.org/mailman/listinfo/buildroot -- .-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 561 099 427 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------' _______________________________________________ buildroot mailing list buildroot@buildroot.org https://lists.buildroot.org/mailman/listinfo/buildroot ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [Buildroot] [PATCH 2/3] support/scripts: don't require gawk to generate glibc gconv modules 2022-10-14 11:18 ` [Buildroot] [PATCH 2/3] support/scripts: don't require gawk to generate glibc gconv modules yann.morin 2022-10-21 19:08 ` Yann E. MORIN @ 2022-11-04 7:40 ` Peter Korsgaard 1 sibling, 0 replies; 10+ messages in thread From: Peter Korsgaard @ 2022-11-04 7:40 UTC (permalink / raw) To: yann.morin; +Cc: buildroot >>>>> <yann.morin@orange.com> writes: > When a subset of the glibc gconv modules installed, we eed to generate a > trimmed-down list of available modules. We currently use gawk for that. > However, we are not using any GNU extension in that awk script, and it > happens to work as expected when using mawk (which has no GNU > extension). > Commit 11c1076db9a5 (toolchain: add option to copy the gconv libraries) > did not explain why it used gawk explicitly, and given the age for that > commit, we doubt we'd be able to have the involved participants recall > anything from that period... > Besides, gawk is not a requirement for Buildroot. > Switch over to using plain awk. > Signed-off-by: Yann E. MORIN <yann.morin@orange.com> Committed to 2022.08.x and 2022.02.x, thanks. -- Bye, Peter Korsgaard _______________________________________________ buildroot mailing list buildroot@buildroot.org https://lists.buildroot.org/mailman/listinfo/buildroot ^ permalink raw reply [flat|nested] 10+ messages in thread
* [Buildroot] [PATCH 3/3] toolchain: suppot gconv modules fro mglibc >= 2.34 [not found] <2b0b6b796ff36967c963ba90f138a03d467ee5cf.1665746266.git.yann.morin@orange.com> 2022-10-14 11:18 ` [Buildroot] [PATCH 2/3] support/scripts: don't require gawk to generate glibc gconv modules yann.morin @ 2022-10-14 11:18 ` yann.morin 2022-10-21 19:10 ` Yann E. MORIN 2022-11-04 7:43 ` Peter Korsgaard 1 sibling, 2 replies; 10+ messages in thread From: yann.morin @ 2022-10-14 11:18 UTC (permalink / raw) To: buildroot Cc: Giulio Benetti, Romain Naour, yann.morin, Thomas De Schampheleire, Thomas Petazzoni Startig with glibc 2.34, the gconv modules description has been split in two: - a common definition in the old location, /usr/lib/gconv/gconv-modules - specific defitions in a subdirectory, /usr/lib/gconv/gconv-modules.d/ This is done so as to simplify the handling of glibc gconv modules, and eventually to segregate those outside of glibc, and so that third-parties may also provide their own gconv converters and their definitions. And starting with that same glibc version, most of the gconv modules definition is moved to an extra configuration file in that sub-directory. It is thus no longer possible to use special code pages, like cp850, which are very usefull to access FAT-formatted devices. Add suppot for this new gconv layout, while keeping support for older glibc versions. Note that the modules themselves are not moved or renaed, just the definition files have changed. Instead of passing the one old gonv modules definitions file on stdin, we pass the base directory to that file, and move into the script the responsibility to find all the gconv definition files. Signed-off-by: Yann E. MORIN <yann.morin@orange.com> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Cc: Romain Naour <romain.naour@gmail.com> Cc: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com> Cc: Giulio Benetti <giulio.benetti@benettiengineering.com> --- support/scripts/expunge-gconv-modules | 22 ++++++++++++++++------ toolchain/toolchain.mk | 9 +++++++-- 2 files changed, 23 insertions(+), 8 deletions(-) diff --git a/support/scripts/expunge-gconv-modules b/support/scripts/expunge-gconv-modules index bc60fc0ce4..e9ac48ca3f 100755 --- a/support/scripts/expunge-gconv-modules +++ b/support/scripts/expunge-gconv-modules @@ -1,11 +1,17 @@ #!/usr/bin/env bash # This script is used to generate a gconv-modules file that takes into -# account only the gconv modules installed by Buildroot. It receives -# on its standard input the original complete gconv-modules file from -# the toolchain, and as arguments the list of gconv modules that were -# actually installed, and writes on its standard output the new -# gconv-modules file. +# account only the gconv modules installed by Buildroot, and generates +# a stripped-down gconv-moduels file on its stdout. +# It takes two arguments: +# $1: the directory where to look for gconv modules definitions +# $2: a space-separated list of gconv modules that were actually +# installed + +# Starting with glibc-2.34, modules definitions are located in multiple +# files: +# ${1}/gconv-modules +# ${1}/gconv-modules.d/*.conf # The format of gconv-modules is precisely documented in the # file itself. It consists of two different directives: @@ -19,7 +25,11 @@ # we handle each with slightly different code, since the second never has # associated aliases. -awk -v files="${2}" ' +for f in ${1}/gconv-modules ${1}/gconv-modules.d/*.conf; do + [ -e "${f}" ] || continue + cat "${f}" +done \ +|awk -v files="${2}" ' $1 == "alias" { aliases[$3] = aliases[$3] " " $2; } diff --git a/toolchain/toolchain.mk b/toolchain/toolchain.mk index 08d1649603..fe87a72ed4 100644 --- a/toolchain/toolchain.mk +++ b/toolchain/toolchain.mk @@ -27,6 +27,10 @@ define TOOLCHAIN_GLIBC_COPY_GCONV_LIBS $(INSTALL) -m 0644 $(STAGING_DIR)/usr/lib/$${d}/gconv/*.so \ $(TARGET_DIR)/usr/lib/gconv \ || exit 1; \ + if [ -d $(STAGING_DIR)/usr/lib/$${d}/gconv/gconv-modules.d ]; then \ + cp -a $(STAGING_DIR)/usr/lib/$${d}/gconv/gconv-modules.d \ + $(TARGET_DIR)/usr/lib/gconv/ || exit 1; \ + fi; \ else \ for l in $(TOOLCHAIN_GLIBC_GCONV_LIBS); do \ $(INSTALL) -m 0644 -D $(STAGING_DIR)/usr/lib/$${d}/gconv/$${l}.so \ @@ -41,8 +45,9 @@ define TOOLCHAIN_GLIBC_COPY_GCONV_LIBS || exit 1; \ done; \ done; \ - ./support/scripts/expunge-gconv-modules "$(TOOLCHAIN_GLIBC_GCONV_LIBS)" \ - <$(STAGING_DIR)/usr/lib/$${d}/gconv/gconv-modules \ + ./support/scripts/expunge-gconv-modules \ + $(STAGING_DIR)/usr/lib/$${d}/gconv \ + "$(TOOLCHAIN_GLIBC_GCONV_LIBS)" \ >$(TARGET_DIR)/usr/lib/gconv/gconv-modules; \ fi endef -- 2.25.1 _________________________________________________________________________________________________________________________ Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. If you have received this email in error, please notify the sender and delete this message and its attachments. As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified. Thank you. _______________________________________________ buildroot mailing list buildroot@buildroot.org https://lists.buildroot.org/mailman/listinfo/buildroot ^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: [Buildroot] [PATCH 3/3] toolchain: suppot gconv modules fro mglibc >= 2.34 2022-10-14 11:18 ` [Buildroot] [PATCH 3/3] toolchain: suppot gconv modules fro mglibc >= 2.34 yann.morin @ 2022-10-21 19:10 ` Yann E. MORIN 2022-11-04 7:43 ` Peter Korsgaard 1 sibling, 0 replies; 10+ messages in thread From: Yann E. MORIN @ 2022-10-21 19:10 UTC (permalink / raw) To: yann.morin Cc: Thomas Petazzoni, Giulio Benetti, Romain Naour, Thomas De Schampheleire, buildroot Yann, All, In the title: *support, from glibc Also, many typoes in the commit log... On 2022-10-14 13:18 +0200, yann.morin@orange.com spake thusly: > Startig with glibc 2.34, the gconv modules description has been split in > two: > - a common definition in the old location, /usr/lib/gconv/gconv-modules > - specific defitions in a subdirectory, /usr/lib/gconv/gconv-modules.d/ > > This is done so as to simplify the handling of glibc gconv modules, and > eventually to segregate those outside of glibc, and so that third-parties > may also provide their own gconv converters and their definitions. > > And starting with that same glibc version, most of the gconv modules > definition is moved to an extra configuration file in that > sub-directory. > > It is thus no longer possible to use special code pages, like cp850, > which are very usefull to access FAT-formatted devices. > > Add suppot for this new gconv layout, while keeping support for older > glibc versions. Note that the modules themselves are not moved or > renaed, just the definition files have changed. > > Instead of passing the one old gonv modules definitions file on stdin, > we pass the base directory to that file, and move into the script the > responsibility to find all the gconv definition files. > > Signed-off-by: Yann E. MORIN <yann.morin@orange.com> > Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> > Cc: Romain Naour <romain.naour@gmail.com> > Cc: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com> > Cc: Giulio Benetti <giulio.benetti@benettiengineering.com> Applied to master with the many typoes fixed, thanks. Regards, Yann E. MORIN. > --- > support/scripts/expunge-gconv-modules | 22 ++++++++++++++++------ > toolchain/toolchain.mk | 9 +++++++-- > 2 files changed, 23 insertions(+), 8 deletions(-) > > diff --git a/support/scripts/expunge-gconv-modules b/support/scripts/expunge-gconv-modules > index bc60fc0ce4..e9ac48ca3f 100755 > --- a/support/scripts/expunge-gconv-modules > +++ b/support/scripts/expunge-gconv-modules > @@ -1,11 +1,17 @@ > #!/usr/bin/env bash > > # This script is used to generate a gconv-modules file that takes into > -# account only the gconv modules installed by Buildroot. It receives > -# on its standard input the original complete gconv-modules file from > -# the toolchain, and as arguments the list of gconv modules that were > -# actually installed, and writes on its standard output the new > -# gconv-modules file. > +# account only the gconv modules installed by Buildroot, and generates > +# a stripped-down gconv-moduels file on its stdout. > +# It takes two arguments: > +# $1: the directory where to look for gconv modules definitions > +# $2: a space-separated list of gconv modules that were actually > +# installed > + > +# Starting with glibc-2.34, modules definitions are located in multiple > +# files: > +# ${1}/gconv-modules > +# ${1}/gconv-modules.d/*.conf > > # The format of gconv-modules is precisely documented in the > # file itself. It consists of two different directives: > @@ -19,7 +25,11 @@ > # we handle each with slightly different code, since the second never has > # associated aliases. > > -awk -v files="${2}" ' > +for f in ${1}/gconv-modules ${1}/gconv-modules.d/*.conf; do > + [ -e "${f}" ] || continue > + cat "${f}" > +done \ > +|awk -v files="${2}" ' > $1 == "alias" { > aliases[$3] = aliases[$3] " " $2; > } > diff --git a/toolchain/toolchain.mk b/toolchain/toolchain.mk > index 08d1649603..fe87a72ed4 100644 > --- a/toolchain/toolchain.mk > +++ b/toolchain/toolchain.mk > @@ -27,6 +27,10 @@ define TOOLCHAIN_GLIBC_COPY_GCONV_LIBS > $(INSTALL) -m 0644 $(STAGING_DIR)/usr/lib/$${d}/gconv/*.so \ > $(TARGET_DIR)/usr/lib/gconv \ > || exit 1; \ > + if [ -d $(STAGING_DIR)/usr/lib/$${d}/gconv/gconv-modules.d ]; then \ > + cp -a $(STAGING_DIR)/usr/lib/$${d}/gconv/gconv-modules.d \ > + $(TARGET_DIR)/usr/lib/gconv/ || exit 1; \ > + fi; \ > else \ > for l in $(TOOLCHAIN_GLIBC_GCONV_LIBS); do \ > $(INSTALL) -m 0644 -D $(STAGING_DIR)/usr/lib/$${d}/gconv/$${l}.so \ > @@ -41,8 +45,9 @@ define TOOLCHAIN_GLIBC_COPY_GCONV_LIBS > || exit 1; \ > done; \ > done; \ > - ./support/scripts/expunge-gconv-modules "$(TOOLCHAIN_GLIBC_GCONV_LIBS)" \ > - <$(STAGING_DIR)/usr/lib/$${d}/gconv/gconv-modules \ > + ./support/scripts/expunge-gconv-modules \ > + $(STAGING_DIR)/usr/lib/$${d}/gconv \ > + "$(TOOLCHAIN_GLIBC_GCONV_LIBS)" \ > >$(TARGET_DIR)/usr/lib/gconv/gconv-modules; \ > fi > endef > -- > 2.25.1 > > > _________________________________________________________________________________________________________________________ > > Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc > pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler > a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, > Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. > > This message and its attachments may contain confidential or privileged information that may be protected by law; > they should not be distributed, used or copied without authorisation. > If you have received this email in error, please notify the sender and delete this message and its attachments. > As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified. > Thank you. > > _______________________________________________ > buildroot mailing list > buildroot@buildroot.org > https://lists.buildroot.org/mailman/listinfo/buildroot -- .-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 561 099 427 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------' _______________________________________________ buildroot mailing list buildroot@buildroot.org https://lists.buildroot.org/mailman/listinfo/buildroot ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [Buildroot] [PATCH 3/3] toolchain: suppot gconv modules fro mglibc >= 2.34 2022-10-14 11:18 ` [Buildroot] [PATCH 3/3] toolchain: suppot gconv modules fro mglibc >= 2.34 yann.morin 2022-10-21 19:10 ` Yann E. MORIN @ 2022-11-04 7:43 ` Peter Korsgaard 1 sibling, 0 replies; 10+ messages in thread From: Peter Korsgaard @ 2022-11-04 7:43 UTC (permalink / raw) To: yann.morin Cc: Thomas Petazzoni, Giulio Benetti, Romain Naour, Thomas De Schampheleire, buildroot >>>>> <yann.morin@orange.com> writes: > Startig with glibc 2.34, the gconv modules description has been split in > two: > - a common definition in the old location, /usr/lib/gconv/gconv-modules > - specific defitions in a subdirectory, /usr/lib/gconv/gconv-modules.d/ > This is done so as to simplify the handling of glibc gconv modules, and > eventually to segregate those outside of glibc, and so that third-parties > may also provide their own gconv converters and their definitions. > And starting with that same glibc version, most of the gconv modules > definition is moved to an extra configuration file in that > sub-directory. > It is thus no longer possible to use special code pages, like cp850, > which are very usefull to access FAT-formatted devices. > Add suppot for this new gconv layout, while keeping support for older > glibc versions. Note that the modules themselves are not moved or > renaed, just the definition files have changed. > Instead of passing the one old gonv modules definitions file on stdin, > we pass the base directory to that file, and move into the script the > responsibility to find all the gconv definition files. > Signed-off-by: Yann E. MORIN <yann.morin@orange.com> > Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> > Cc: Romain Naour <romain.naour@gmail.com> > Cc: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com> > Cc: Giulio Benetti <giulio.benetti@benettiengineering.com> Committed to 2022.08.x and 2022.02.x, thanks. -- Bye, Peter Korsgaard _______________________________________________ buildroot mailing list buildroot@buildroot.org https://lists.buildroot.org/mailman/listinfo/buildroot ^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2022-11-04 7:43 UTC | newest] Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2022-10-14 11:18 [Buildroot] [PATCH 1/3] Makefile: really generate glibc locales in parallel yann.morin 2022-10-21 19:07 ` Yann E. MORIN 2022-10-21 19:46 ` Yann E. MORIN 2022-11-03 13:49 ` Peter Korsgaard [not found] <2b0b6b796ff36967c963ba90f138a03d467ee5cf.1665746266.git.yann.morin@orange.com> 2022-10-14 11:18 ` [Buildroot] [PATCH 2/3] support/scripts: don't require gawk to generate glibc gconv modules yann.morin 2022-10-21 19:08 ` Yann E. MORIN 2022-11-04 7:40 ` Peter Korsgaard 2022-10-14 11:18 ` [Buildroot] [PATCH 3/3] toolchain: suppot gconv modules fro mglibc >= 2.34 yann.morin 2022-10-21 19:10 ` Yann E. MORIN 2022-11-04 7:43 ` Peter Korsgaard
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.