[PATCH] glibc: Fix locale DEPENDS

[PATCH] glibc: Fix locale DEPENDS

[Joshua Watt]

gettext is required to generate the glibc locales in do_compile. If not
present, glibc will skip the generation which isn't reproducible.

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
---
 meta/recipes-core/glibc/glibc_2.29.bb | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-core/glibc/glibc_2.29.bb b/meta/recipes-core/glibc/glibc_2.29.bb
index 073d1533e37..c400ee10b0f 100644
--- a/meta/recipes-core/glibc/glibc_2.29.bb
+++ b/meta/recipes-core/glibc/glibc_2.29.bb
@@ -5,7 +5,7 @@ LIC_FILES_CHKSUM = "file://LICENSES;md5=cfc0ed77a9f62fa62eded042ebe31d72 \
       file://posix/rxspencer/COPYRIGHT;md5=dc5485bb394a13b2332ec1c785f5d83a \
       file://COPYING.LIB;md5=4fbd65380cdd255951079008b364516c"
 
-DEPENDS += "gperf-native bison-native make-native"
+DEPENDS += "gperf-native bison-native make-native gettext-native"
 
 PV = "2.29"
 
-- 
2.21.0

Re: [PATCH] glibc: Fix locale DEPENDS

[Burton, Ross]

On Thu, 27 Jun 2019 at 14:55, Joshua Watt <jpewhacker@gmail.com> wrote:
> gettext is required to generate the glibc locales in do_compile. If not
> present, glibc will skip the generation which isn't reproducible.
>
> Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
> ---
>  meta/recipes-core/glibc/glibc_2.29.bb | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/meta/recipes-core/glibc/glibc_2.29.bb b/meta/recipes-core/glibc/glibc_2.29.bb
> index 073d1533e37..c400ee10b0f 100644
> --- a/meta/recipes-core/glibc/glibc_2.29.bb
> +++ b/meta/recipes-core/glibc/glibc_2.29.bb
> @@ -5,7 +5,7 @@ LIC_FILES_CHKSUM = "file://LICENSES;md5=cfc0ed77a9f62fa62eded042ebe31d72 \
>        file://posix/rxspencer/COPYRIGHT;md5=dc5485bb394a13b2332ec1c785f5d83a \
>        file://COPYING.LIB;md5=4fbd65380cdd255951079008b364516c"
>
> -DEPENDS += "gperf-native bison-native make-native"
> +DEPENDS += "gperf-native bison-native make-native gettext-native"

Dont we want this to respect the NLS flag or similar?

Ross

Re: [PATCH] glibc: Fix locale DEPENDS

[Joshua Watt]

On 6/27/19 11:19 AM, Burton, Ross wrote:
> On Thu, 27 Jun 2019 at 14:55, Joshua Watt <jpewhacker@gmail.com> wrote:
>> gettext is required to generate the glibc locales in do_compile. If not
>> present, glibc will skip the generation which isn't reproducible.
>>
>> Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
>> ---
>>   meta/recipes-core/glibc/glibc_2.29.bb | 2 +-
>>   1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/meta/recipes-core/glibc/glibc_2.29.bb b/meta/recipes-core/glibc/glibc_2.29.bb
>> index 073d1533e37..c400ee10b0f 100644
>> --- a/meta/recipes-core/glibc/glibc_2.29.bb
>> +++ b/meta/recipes-core/glibc/glibc_2.29.bb
>> @@ -5,7 +5,7 @@ LIC_FILES_CHKSUM = "file://LICENSES;md5=cfc0ed77a9f62fa62eded042ebe31d72 \
>>         file://posix/rxspencer/COPYRIGHT;md5=dc5485bb394a13b2332ec1c785f5d83a \
>>         file://COPYING.LIB;md5=4fbd65380cdd255951079008b364516c"
>>
>> -DEPENDS += "gperf-native bison-native make-native"
>> +DEPENDS += "gperf-native bison-native make-native gettext-native"
> Dont we want this to respect the NLS flag or similar?

Sorry, I'm not quite sure I follow... I didn't find anything related to 
NLS in the glibc recipes? Would you mind clarifying?

>
> Ross

Re: [PATCH] glibc: Fix locale DEPENDS

[Andre McCurdy]

On Thu, Jun 27, 2019 at 6:54 AM Joshua Watt <jpewhacker@gmail.com> wrote:
>
> gettext is required to generate the glibc locales in do_compile. If not
> present, glibc will skip the generation which isn't reproducible.

I guess you mean not deterministic rather than not reproducible?

> Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
> ---
>  meta/recipes-core/glibc/glibc_2.29.bb | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/meta/recipes-core/glibc/glibc_2.29.bb b/meta/recipes-core/glibc/glibc_2.29.bb
> index 073d1533e37..c400ee10b0f 100644
> --- a/meta/recipes-core/glibc/glibc_2.29.bb
> +++ b/meta/recipes-core/glibc/glibc_2.29.bb
> @@ -5,7 +5,7 @@ LIC_FILES_CHKSUM = "file://LICENSES;md5=cfc0ed77a9f62fa62eded042ebe31d72 \
>        file://posix/rxspencer/COPYRIGHT;md5=dc5485bb394a13b2332ec1c785f5d83a \
>        file://COPYING.LIB;md5=4fbd65380cdd255951079008b364516c"
>
> -DEPENDS += "gperf-native bison-native make-native"
> +DEPENDS += "gperf-native bison-native make-native gettext-native"
>
>  PV = "2.29"
>
> --
> 2.21.0
>
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [PATCH] glibc: Fix locale DEPENDS

[Joshua Watt]

On 6/27/19 2:47 PM, Andre McCurdy wrote:
> On Thu, Jun 27, 2019 at 6:54 AM Joshua Watt <jpewhacker@gmail.com> wrote:
>> gettext is required to generate the glibc locales in do_compile. If not
>> present, glibc will skip the generation which isn't reproducible.
> I guess you mean not deterministic rather than not reproducible?

Correct. I found all these in the context of trying to get reproducible 
builds working, but that is a more accurate statement.

>
>> Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
>> ---
>>   meta/recipes-core/glibc/glibc_2.29.bb | 2 +-
>>   1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/meta/recipes-core/glibc/glibc_2.29.bb b/meta/recipes-core/glibc/glibc_2.29.bb
>> index 073d1533e37..c400ee10b0f 100644
>> --- a/meta/recipes-core/glibc/glibc_2.29.bb
>> +++ b/meta/recipes-core/glibc/glibc_2.29.bb
>> @@ -5,7 +5,7 @@ LIC_FILES_CHKSUM = "file://LICENSES;md5=cfc0ed77a9f62fa62eded042ebe31d72 \
>>         file://posix/rxspencer/COPYRIGHT;md5=dc5485bb394a13b2332ec1c785f5d83a \
>>         file://COPYING.LIB;md5=4fbd65380cdd255951079008b364516c"
>>
>> -DEPENDS += "gperf-native bison-native make-native"
>> +DEPENDS += "gperf-native bison-native make-native gettext-native"
>>
>>   PV = "2.29"
>>
>> --
>> 2.21.0
>>
>> --
>> _______________________________________________
>> Openembedded-core mailing list
>> Openembedded-core@lists.openembedded.org
>> http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [PATCH] glibc: Fix locale DEPENDS

[Khem Raj]

[-- Attachment #1: Type: text/plain, Size: 1296 bytes --]

On Thu, Jun 27, 2019 at 9:54 AM Joshua Watt <jpewhacker@gmail.com> wrote:

> gettext is required to generate the glibc locales in do_compile. If not
> present, glibc will skip the generation which isn't reproducible.
>
> Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
> ---
>  meta/recipes-core/glibc/glibc_2.29.bb | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/meta/recipes-core/glibc/glibc_2.29.bb
> b/meta/recipes-core/glibc/glibc_2.29.bb
> index 073d1533e37..c400ee10b0f 100644
> --- a/meta/recipes-core/glibc/glibc_2.29.bb
> +++ b/meta/recipes-core/glibc/glibc_2.29.bb
> @@ -5,7 +5,7 @@ LIC_FILES_CHKSUM =
> "file://LICENSES;md5=cfc0ed77a9f62fa62eded042ebe31d72 \
>
>  file://posix/rxspencer/COPYRIGHT;md5=dc5485bb394a13b2332ec1c785f5d83a \
>        file://COPYING.LIB;md5=4fbd65380cdd255951079008b364516c"
>
> -DEPENDS += "gperf-native bison-native make-native"
> +DEPENDS += "gperf-native bison-native make-native gettext-native"


I think we have had this dependency proposed in past and one think that
worries me is that it will cause serialization a bit more and can affect
bulls times wasn’t there a minimal gettext at some point and I wonder if
that could satisfy needs here

>
>
>  PV = "2.29"
>
> --
> 2.21.0
>
>

[-- Attachment #2: Type: text/html, Size: 2221 bytes --]

Re: [PATCH] glibc: Fix locale DEPENDS

[Peter Kjellerstedt]

[-- Attachment #1: Type: text/plain, Size: 2049 bytes --]

Are you thinking of gettext-minimal-native? It is used by gettext.bbclass if USE_NLS is “no”.

//Peter

From: openembedded-core-bounces@lists.openembedded.org <openembedded-core-bounces@lists.openembedded.org> On Behalf Of Khem Raj
Sent: den 28 juni 2019 18:37
To: Joshua Watt <jpewhacker@gmail.com>
Cc: openembedded-core@lists.openembedded.org
Subject: Re: [OE-core] [PATCH] glibc: Fix locale DEPENDS



On Thu, Jun 27, 2019 at 9:54 AM Joshua Watt <jpewhacker@gmail.com<mailto:jpewhacker@gmail.com>> wrote:
gettext is required to generate the glibc locales in do_compile. If not
present, glibc will skip the generation which isn't reproducible.

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com<mailto:JPEWhacker@gmail.com>>
---
 meta/recipes-core/glibc/glibc_2.29.bb<http://glibc_2.29.bb> | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-core/glibc/glibc_2.29.bb<http://glibc_2.29.bb> b/meta/recipes-core/glibc/glibc_2.29.bb<http://glibc_2.29.bb>
index 073d1533e37..c400ee10b0f 100644
--- a/meta/recipes-core/glibc/glibc_2.29.bb<http://glibc_2.29.bb>
+++ b/meta/recipes-core/glibc/glibc_2.29.bb<http://glibc_2.29.bb>
@@ -5,7 +5,7 @@ LIC_FILES_CHKSUM = "file://LICENSES;md5=cfc0ed77a9f62fa62eded042ebe31d72 \
       file://posix/rxspencer/COPYRIGHT;md5=dc5485bb394a13b2332ec1c785f5d83a \
       file://COPYING.LIB;md5=4fbd65380cdd255951079008b364516c<file://LICENSES;md5=cfc0ed77a9f62fa62eded042ebe31d72%20/%0b %20 %20 %20 file:/posix/rxspencer/COPYRIGHT;md5=dc5485bb394a13b2332ec1c785f5d83a%20/%0b %20 %20 %20 file:/COPYING.LIB;md5=4fbd65380cdd255951079008b364516c>"

-DEPENDS += "gperf-native bison-native make-native"
+DEPENDS += "gperf-native bison-native make-native gettext-native"

I think we have had this dependency proposed in past and one think that worries me is that it will cause serialization a bit more and can affect bulls times wasn’t there a minimal gettext at some point and I wonder if that could satisfy needs here


 PV = "2.29"

--
2.21.0

[-- Attachment #2: Type: text/html, Size: 5981 bytes --]

Re: [PATCH] glibc: Fix locale DEPENDS

[Khem Raj]

[-- Attachment #1: Type: text/plain, Size: 1908 bytes --]

Yes, I think so.

On Mon, Jul 1, 2019 at 7:13 PM Peter Kjellerstedt <
peter.kjellerstedt@axis.com> wrote:

> Are you thinking of gettext-minimal-native? It is used by gettext.bbclass
> if USE_NLS is “no”.
>
>
>
> //Peter
>
>
>
> *From:* openembedded-core-bounces@lists.openembedded.org <
> openembedded-core-bounces@lists.openembedded.org> *On Behalf Of *Khem Raj
> *Sent:* den 28 juni 2019 18:37
> *To:* Joshua Watt <jpewhacker@gmail.com>
> *Cc:* openembedded-core@lists.openembedded.org
> *Subject:* Re: [OE-core] [PATCH] glibc: Fix locale DEPENDS
>
>
>
>
>
>
>
> On Thu, Jun 27, 2019 at 9:54 AM Joshua Watt <jpewhacker@gmail.com> wrote:
>
> gettext is required to generate the glibc locales in do_compile. If not
> present, glibc will skip the generation which isn't reproducible.
>
> Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
> ---
>  meta/recipes-core/glibc/glibc_2.29.bb | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/meta/recipes-core/glibc/glibc_2.29.bb
> b/meta/recipes-core/glibc/glibc_2.29.bb
> index 073d1533e37..c400ee10b0f 100644
> --- a/meta/recipes-core/glibc/glibc_2.29.bb
> +++ b/meta/recipes-core/glibc/glibc_2.29.bb
> @@ -5,7 +5,7 @@ LIC_FILES_CHKSUM = "file://LICENSES;md5=cfc0ed77a9f62fa62eded042ebe31d72
> \
>
>  file://posix/rxspencer/COPYRIGHT;md5=dc5485bb394a13b2332ec1c785f5d83a \
>        file://COPYING.LIB;md5=4fbd65380cdd255951079008b364516c"
>
> -DEPENDS += "gperf-native bison-native make-native"
> +DEPENDS += "gperf-native bison-native make-native gettext-native"
>
>
>
> I think we have had this dependency proposed in past and one think that
> worries me is that it will cause serialization a bit more and can affect
> bulls times wasn’t there a minimal gettext at some point and I wonder if
> that could satisfy needs here
>
>
>
>  PV = "2.29"
>
> --
> 2.21.0
>
>

[-- Attachment #2: Type: text/html, Size: 4611 bytes --]

Re: [PATCH] glibc: Fix locale DEPENDS

[Joshua Watt]

On Mon, Jul 1, 2019 at 10:25 PM Khem Raj <raj.khem@gmail.com> wrote:
>
> Yes, I think so.

Oops, I forgot to mark the latest patch in-reply-to this one. It's
here: http://lists.openembedded.org/pipermail/openembedded-core/2019-August/286123.html

>
> On Mon, Jul 1, 2019 at 7:13 PM Peter Kjellerstedt <peter.kjellerstedt@axis.com> wrote:
>>
>> Are you thinking of gettext-minimal-native? It is used by gettext.bbclass if USE_NLS is “no”.
>>
>>
>>
>> //Peter
>>
>>
>>
>> From: openembedded-core-bounces@lists.openembedded.org <openembedded-core-bounces@lists.openembedded.org> On Behalf Of Khem Raj
>> Sent: den 28 juni 2019 18:37
>> To: Joshua Watt <jpewhacker@gmail.com>
>> Cc: openembedded-core@lists.openembedded.org
>> Subject: Re: [OE-core] [PATCH] glibc: Fix locale DEPENDS
>>
>>
>>
>>
>>
>>
>>
>> On Thu, Jun 27, 2019 at 9:54 AM Joshua Watt <jpewhacker@gmail.com> wrote:
>>
>> gettext is required to generate the glibc locales in do_compile. If not
>> present, glibc will skip the generation which isn't reproducible.
>>
>> Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
>> ---
>>  meta/recipes-core/glibc/glibc_2.29.bb | 2 +-
>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/meta/recipes-core/glibc/glibc_2.29.bb b/meta/recipes-core/glibc/glibc_2.29.bb
>> index 073d1533e37..c400ee10b0f 100644
>> --- a/meta/recipes-core/glibc/glibc_2.29.bb
>> +++ b/meta/recipes-core/glibc/glibc_2.29.bb
>> @@ -5,7 +5,7 @@ LIC_FILES_CHKSUM = "file://LICENSES;md5=cfc0ed77a9f62fa62eded042ebe31d72 \
>>        file://posix/rxspencer/COPYRIGHT;md5=dc5485bb394a13b2332ec1c785f5d83a \
>>        file://COPYING.LIB;md5=4fbd65380cdd255951079008b364516c"
>>
>> -DEPENDS += "gperf-native bison-native make-native"
>> +DEPENDS += "gperf-native bison-native make-native gettext-native"
>>
>>
>>
>> I think we have had this dependency proposed in past and one think that worries me is that it will cause serialization a bit more and can affect bulls times wasn’t there a minimal gettext at some point and I wonder if that could satisfy needs here
>>
>>
>>
>>  PV = "2.29"
>>
>> --
>> 2.21.0

[PATCH] glibc: Fix locale DEPENDS

[Joshua Watt]

gettext is required to generate the glibc locales in do_compile. If not
present, glibc will skip the generation which isn't reproducible.

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
---
 meta/recipes-core/glibc/glibc.inc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-core/glibc/glibc.inc b/meta/recipes-core/glibc/glibc.inc
index 252fd56c13c..f1a6ae2a245 100644
--- a/meta/recipes-core/glibc/glibc.inc
+++ b/meta/recipes-core/glibc/glibc.inc
@@ -6,7 +6,7 @@ DEPENDS = "virtual/${TARGET_PREFIX}gcc libgcc-initial linux-libc-headers"
 
 PROVIDES = "virtual/libc"
 PROVIDES += "virtual/libintl virtual/libiconv"
-inherit autotools texinfo distro_features_check systemd
+inherit autotools texinfo distro_features_check systemd gettext
 
 LEAD_SONAME = "libc.so"
 
-- 
2.21.0

Re: [PATCH] glibc: Fix locale DEPENDS

[Richard Purdie]

On Tue, 2019-08-27 at 07:45 -0500, Joshua Watt wrote:
> gettext is required to generate the glibc locales in do_compile. If not
> present, glibc will skip the generation which isn't reproducible.
> 
> Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
> ---
>  meta/recipes-core/glibc/glibc.inc | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/meta/recipes-core/glibc/glibc.inc b/meta/recipes-core/glibc/glibc.inc
> index 252fd56c13c..f1a6ae2a245 100644
> --- a/meta/recipes-core/glibc/glibc.inc
> +++ b/meta/recipes-core/glibc/glibc.inc
> @@ -6,7 +6,7 @@ DEPENDS = "virtual/${TARGET_PREFIX}gcc libgcc-initial linux-libc-headers"
>  
>  PROVIDES = "virtual/libc"
>  PROVIDES += "virtual/libintl virtual/libiconv"
> -inherit autotools texinfo distro_features_check systemd
> +inherit autotools texinfo distro_features_check systemd gettext

I suspect this may not do what you expect.

At least as I read the class and recipes, glibc sets
INHIBIT_DEFAULT_DEPS which means no gettext-native dependency is added,
instead, configure has --disable-nls added.

Is that what we want?

I'm curious to understand how glibc locales are ever generated
correctly and what we're aiming to add here (a gettext-native
dependency?)

FWIW gettext-native is a very heavy thing to add in as a dependency
from a build time perspective.

Cheers,

Richard

Re: [PATCH] glibc: Fix locale DEPENDS

[Joshua Watt]

On 8/28/19 11:09 AM, Richard Purdie wrote:
> On Tue, 2019-08-27 at 07:45 -0500, Joshua Watt wrote:
>> gettext is required to generate the glibc locales in do_compile. If not
>> present, glibc will skip the generation which isn't reproducible.
>>
>> Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
>> ---
>>   meta/recipes-core/glibc/glibc.inc | 2 +-
>>   1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/meta/recipes-core/glibc/glibc.inc b/meta/recipes-core/glibc/glibc.inc
>> index 252fd56c13c..f1a6ae2a245 100644
>> --- a/meta/recipes-core/glibc/glibc.inc
>> +++ b/meta/recipes-core/glibc/glibc.inc
>> @@ -6,7 +6,7 @@ DEPENDS = "virtual/${TARGET_PREFIX}gcc libgcc-initial linux-libc-headers"
>>   
>>   PROVIDES = "virtual/libc"
>>   PROVIDES += "virtual/libintl virtual/libiconv"
>> -inherit autotools texinfo distro_features_check systemd
>> +inherit autotools texinfo distro_features_check systemd gettext

FYI: I forgot to send this one in-reply-to a previous patch. There is 
more history here: 
http://lists.openembedded.org/pipermail/openembedded-core/2019-June/284158.html

> I suspect this may not do what you expect.
>
> At least as I read the class and recipes, glibc sets
> INHIBIT_DEFAULT_DEPS which means no gettext-native dependency is added,
> instead, configure has --disable-nls added.
Ah, yes I see that also.
>
> Is that what we want?

I guess I'm not sure exactly what we want; It was a while ago so I don't 
remember exactly what glibc was doing, but it looks like it was 
sometimes generating locales (e.g. the first time it built) and 
sometimes not (when rebuilding) depending on the presence of gettext.

>
> I'm curious to understand how glibc locales are ever generated
> correctly and what we're aiming to add here (a gettext-native
> dependency?)

I'm not sure how it ever works correctly :). Perhaps it would be best to 
just drop this patch for now until we take a more serious look at 
reproducible in the presence of partial recipe rebuilds. I'd rather 
focus on getting the existing QA test running ATM.

>
> FWIW gettext-native is a very heavy thing to add in as a dependency
> from a build time perspective.
Yes, that was discussed which is what led to using gettext.bbclass that 
might use gettext-minimal-native, but as you pointed out this isn't 
whats happening.
>
> Cheers,
>
> Richard
>