Re: [oe-commits] [openembedded-core] 02/09: glib-2.0: add missing locale dependencies to PN-ptest

Re: [oe-commits] [openembedded-core] 02/09: glib-2.0: add missing locale dependencies to PN-ptest

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 2778 bytes --]

This is causing a lot of QA errors like:

lib32-glib-2.0-2.58.0: lib32-glib-2.0-ptest rdepends on
lib32-locale-base-ru-ru, but it isn't a build dependency? [build-deps]
lib32-glib-2.0-2.58.0: lib32-glib-2.0-ptest rdepends on
lib32-locale-base-tr-tr, but it isn't a build dependency? [build-deps]

On Wed, Oct 10, 2018 at 2:18 PM <git@git.openembedded.org> wrote:

> This is an automated email from the git hooks/post-receive script.
>
> rpurdie pushed a commit to branch master-next
> in repository openembedded-core.
>
> commit f1c122fdc14624a659f8d4b07e1ff1f21e472e50
> Author: Ross Burton <ross.burton@intel.com>
> AuthorDate: Tue Oct 9 12:00:19 2018 +0100
>
>     glib-2.0: add missing locale dependencies to PN-ptest
>
>     The test suite skips several tests if the required locales are not
> found, so add
>     them as dependencies of glib-2.0-ptest.
>
>     (From OE-Core rev: b9317997bd4527b7873eb4f903cc4890b5b3580f)
>
>     Signed-off-by: Ross Burton <ross.burton@intel.com>
>     Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
> ---
>  meta/recipes-core/glib-2.0/glib.inc | 14 ++++++++++++++
>  1 file changed, 14 insertions(+)
>
> diff --git a/meta/recipes-core/glib-2.0/glib.inc
> b/meta/recipes-core/glib-2.0/glib.inc
> index ad4f2ae..92dcf99 100644
> --- a/meta/recipes-core/glib-2.0/glib.inc
> +++ b/meta/recipes-core/glib-2.0/glib.inc
> @@ -140,6 +140,8 @@ RDEPENDS_${PN}-ptest += "\
>              shared-mime-info \
>             "
>
> +# When https://gitlab.gnome.org/GNOME/glib/issues/1343 is
> +# fixed add locale-base-pl-pl
>  RDEPENDS_${PN}-ptest_append_libc-glibc = "\
>              glibc-gconv-utf-16 \
>              glibc-charmap-utf-8 \
> @@ -152,6 +154,18 @@ RDEPENDS_${PN}-ptest_append_libc-glibc = "\
>              glibc-gconv-iso8859-15 \
>              glibc-charmap-invariant \
>              glibc-localedata-translit-cjk-variants \
> +            locale-base-tr-tr \
> +            locale-base-lt-lt \
> +            locale-base-ja-jp.euc-jp \
> +            locale-base-fa-ir \
> +            locale-base-ru-ru \
> +            locale-base-de-de \
> +            locale-base-hr-hr \
> +            locale-base-el-gr \
> +            locale-base-fr-fr \
> +            locale-base-fr-fr \
> +            locale-base-es-es \
> +            locale-base-en-gb \
>             "
>
>  INSANE_SKIP_${PN}-ptest += "libdir"
>
> --
> To stop receiving notification emails like this one, please contact
> the administrator of this repository.
> --
> _______________________________________________
> Openembedded-commits mailing list
> Openembedded-commits@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-commits
>

[-- Attachment #2: Type: text/html, Size: 3894 bytes --]

Re: [oe-commits] [openembedded-core] 02/09: glib-2.0: add missing locale dependencies to PN-ptest

[Burton, Ross]

On Thu, 11 Oct 2018 at 07:38, Martin Jansa <martin.jansa@gmail.com> wrote:
> This is causing a lot of QA errors like:
>
> lib32-glib-2.0-2.58.0: lib32-glib-2.0-ptest rdepends on lib32-locale-base-ru-ru, but it isn't a build dependency? [build-deps]
> lib32-glib-2.0-2.58.0: lib32-glib-2.0-ptest rdepends on lib32-locale-base-tr-tr, but it isn't a build dependency? [build-deps]

Can you test that adding this to the recipe solves the problem:

DEPENDS_append_libc-glibc = " virtual/libc-locale"

Ross

Re: [oe-commits] [openembedded-core] 02/09: glib-2.0: add missing locale dependencies to PN-ptest

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 3742 bytes --]

Hi,

sorry for delay, I was first testing a bit more specific fix:
DEPENDS_append_libc-glibc = "${@bb.utils.contains('PTEST_ENABLED', '1', '',
' virtual/libc-locale', d)}"

but unfortunately this nor your shorter version work.

and it's reproducible without the multilib enabled as well:

ERROR: glib-2.0-1_2.58.0-r0 do_package_qa: QA Issue: glib-2.0-ptest
rdepends on locale-base-de-de, but it isn't a build dependency? [build-deps]
ERROR: glib-2.0-1_2.58.0-r0 do_package_qa: QA Issue: glib-2.0-ptest
rdepends on locale-base-el-gr, but it isn't a build dependency? [build-deps]
ERROR: glib-2.0-1_2.58.0-r0 do_package_qa: QA Issue: glib-2.0-ptest
rdepends on locale-base-en-gb, but it isn't a build dependency? [build-deps]
ERROR: glib-2.0-1_2.58.0-r0 do_package_qa: QA Issue: glib-2.0-ptest
rdepends on locale-base-es-es, but it isn't a build dependency? [build-deps]
ERROR: glib-2.0-1_2.58.0-r0 do_package_qa: QA Issue: glib-2.0-ptest
rdepends on locale-base-fa-ir, but it isn't a build dependency? [build-deps]
ERROR: glib-2.0-1_2.58.0-r0 do_package_qa: QA Issue: glib-2.0-ptest
rdepends on locale-base-fr-fr, but it isn't a build dependency? [build-deps]
ERROR: glib-2.0-1_2.58.0-r0 do_package_qa: QA Issue: glib-2.0-ptest
rdepends on locale-base-hr-hr, but it isn't a build dependency? [build-deps]
ERROR: glib-2.0-1_2.58.0-r0 do_package_qa: QA Issue: glib-2.0-ptest
rdepends on locale-base-ja-jp.euc-jp, but it isn't a build dependency?
[build-deps]
ERROR: glib-2.0-1_2.58.0-r0 do_package_qa: QA Issue: glib-2.0-ptest
rdepends on locale-base-lt-lt, but it isn't a build dependency? [build-deps]
ERROR: glib-2.0-1_2.58.0-r0 do_package_qa: QA Issue: glib-2.0-ptest
rdepends on locale-base-ru-ru, but it isn't a build dependency? [build-deps]
ERROR: glib-2.0-1_2.58.0-r0 do_package_qa: QA Issue: glib-2.0-ptest
rdepends on locale-base-tr-tr, but it isn't a build dependency? [build-deps]
ERROR: glib-2.0-1_2.58.0-r0 do_package_qa: QA run found fatal errors.
Please consider fixing them.
ERROR: glib-2.0-1_2.58.0-r0 do_package_qa: Function failed: do_package_qa

strange, because we had the same issue with bash long time ago and:
commit 2820a2c530018f59895fb57770caa8511d31afd4
Author: Ross Burton <ross.burton@intel.com>
Date:   Fri Oct 13 14:52:16 2017 +0100

    bash: add missing build dependencies for ptest

    Signed-off-by: Ross Burton <ross.burton@intel.com>

diff --git a/meta/recipes-extended/bash/bash.inc
b/meta/recipes-extended/bash/bash.inc
index f8a92b59d0..f4e1f7a11f 100644
--- a/meta/recipes-extended/bash/bash.inc
+++ b/meta/recipes-extended/bash/bash.inc
@@ -24,7 +24,9 @@ ALTERNATIVE_PRIORITY = "100"
 RDEPENDS_${PN} += "base-files"
 RDEPENDS_${PN}_class-nativesdk = ""
 RDEPENDS_${PN}-ptest += "make"
-RDEPENDS_${PN}-ptest_append_libc-glibc += " locale-base-fr-fr
locale-base-de-de"
+
+DEPENDS_append_libc-glibc = " glibc-locale"
+RDEPENDS_${PN}-ptest_append_libc-glibc = " locale-base-fr-fr
locale-base-de-de"

 USERADD_PACKAGES = "${PN}-ptest"
 USERADD_PARAM_${PN}-ptest = "--create-home --user-group test"


worked for bash

On Thu, Oct 11, 2018 at 1:45 PM Burton, Ross <ross.burton@intel.com> wrote:

> On Thu, 11 Oct 2018 at 07:38, Martin Jansa <martin.jansa@gmail.com> wrote:
> > This is causing a lot of QA errors like:
> >
> > lib32-glib-2.0-2.58.0: lib32-glib-2.0-ptest rdepends on
> lib32-locale-base-ru-ru, but it isn't a build dependency? [build-deps]
> > lib32-glib-2.0-2.58.0: lib32-glib-2.0-ptest rdepends on
> lib32-locale-base-tr-tr, but it isn't a build dependency? [build-deps]
>
> Can you test that adding this to the recipe solves the problem:
>
> DEPENDS_append_libc-glibc = " virtual/libc-locale"
>
> Ross
>

[-- Attachment #2: Type: text/html, Size: 4940 bytes --]

Re: [oe-commits] [openembedded-core] 02/09: glib-2.0: add missing locale dependencies to PN-ptest

[Richard Purdie]

On Sat, 2018-10-13 at 02:15 +0200, Martin Jansa wrote:
> Hi,
> 
> sorry for delay, I was first testing a bit more specific fix:
> DEPENDS_append_libc-glibc = "${@bb.utils.contains('PTEST_ENABLED',
> '1', '', ' virtual/libc-locale', d)}"
> 
> but unfortunately this nor your shorter version work.
> 
> and it's reproducible without the multilib enabled as well:
> 
> ERROR: glib-2.0-1_2.58.0-r0 do_package_qa: QA Issue: glib-2.0-ptest rdepends on locale-base-de-de, but it isn't a build dependency? [build-deps]
> ERROR: glib-2.0-1_2.58.0-r0 do_package_qa: QA Issue: glib-2.0-ptest rdepends on locale-base-el-gr, but it isn't a build dependency? [build-deps]
> ERROR: glib-2.0-1_2.58.0-r0 do_package_qa: QA Issue: glib-2.0-ptest rdepends on locale-base-en-gb, but it isn't a build dependency? [build-deps]

What is also puzzling me is that we're not seeing this in OE-Core on
the autobuilders. We recently merged much better warning summary code
so we should see this quite clearly in its output and we're not...

I'm not sure what that means but it suggests there is something going
on here we're not quite seeing...

Cheers,

Richard

Re: [oe-commits] [openembedded-core] 02/09: glib-2.0: add missing locale dependencies to PN-ptest

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 13111 bytes --]

I was checking my builds and here is what I've found.

The dependency on virtual/libc-locale is correctly shown in bitbake -e
glib-2.0.

The glibc-locale is selected as provider for these as expected:

bitbake -v -v -v glib-2.0
...
NOTE: selecting glibc-locale to satisfy virtual/libc-locale due to
PREFERRED_PROVIDERS
NOTE: selecting glibc to satisfy virtual/libc due to PREFERRED_PROVIDERS
NOTE: selecting pkgconfig-native to satisfy pkgconfig-native due to
PREFERRED_PROVIDERS
NOTE: selecting glibc-locale to satisfy runtime locale-base-es-es due to
PREFERRED_PROVIDER_virtual/libc-locale = glibc-locale
NOTE: selecting glibc-locale to satisfy runtime locale-base-fa-ir due to
PREFERRED_PROVIDER_virtual/libc-locale = glibc-locale
NOTE: selecting glibc-locale to satisfy runtime locale-base-de-de due to
PREFERRED_PROVIDER_virtual/libc-locale = glibc-locale
NOTE: selecting glibc-locale to satisfy runtime glibc-gconv-iso8859-15 due
to PREFERRED_PROVIDER_virtual/libc-locale = glibc-locale
NOTE: selecting glibc-locale to satisfy runtime
glibc-localedata-translit-cjk-variants due to
PREFERRED_PROVIDER_virtual/libc-locale = glibc-locale
NOTE: selecting glibc-locale to satisfy runtime glibc-gconv-cp1255 due to
PREFERRED_PROVIDER_virtual/libc-locale = glibc-locale
NOTE: selecting glibc-locale to satisfy runtime glibc-charmap-invariant due
to PREFERRED_PROVIDER_virtual/libc-locale = glibc-locale
NOTE: selecting glibc-locale to satisfy runtime glibc-charmap-utf-8 due to
PREFERRED_PROVIDER_virtual/libc-locale = glibc-locale
NOTE: selecting glibc-locale to satisfy runtime glibc-gconv-utf-32 due to
PREFERRED_PROVIDER_virtual/libc-locale = glibc-locale
NOTE: selecting glibc-locale to satisfy runtime glibc-gconv-utf-16 due to
PREFERRED_PROVIDER_virtual/libc-locale = glibc-locale
NOTE: selecting glibc-locale to satisfy runtime locale-base-en-gb due to
PREFERRED_PROVIDER_virtual/libc-locale = glibc-locale
NOTE: selecting glibc-locale to satisfy runtime glibc-charmap-cp1255 due to
PREFERRED_PROVIDER_virtual/libc-locale = glibc-locale
NOTE: selecting glibc-locale to satisfy runtime locale-base-ru-ru due to
PREFERRED_PROVIDER_virtual/libc-locale = glibc-locale
NOTE: selecting glibc-locale to satisfy runtime locale-base-tr-tr due to
PREFERRED_PROVIDER_virtual/libc-locale = glibc-locale
NOTE: selecting glibc-locale to satisfy runtime locale-base-lt-lt due to
PREFERRED_PROVIDER_virtual/libc-locale = glibc-locale
NOTE: selecting glibc-locale to satisfy runtime locale-base-ja-jp.euc-jp
due to PREFERRED_PROVIDER_virtual/libc-locale = glibc-locale
NOTE: selecting glibc-locale to satisfy runtime locale-base-hr-hr due to
PREFERRED_PROVIDER_virtual/libc-locale = glibc-locale
NOTE: selecting glibc-locale to satisfy runtime locale-base-fr-fr due to
PREFERRED_PROVIDER_virtual/libc-locale = glibc-locale
NOTE: selecting glibc-locale to satisfy runtime glibc-gconv-utf-7 due to
PREFERRED_PROVIDER_virtual/libc-locale = glibc-locale
NOTE: selecting glibc-locale to satisfy runtime glibc-gconv-euc-jp due to
PREFERRED_PROVIDER_virtual/libc-locale = glibc-locale
NOTE: selecting glibc-locale to satisfy runtime glibc-gconv-iso8859-1 due
to PREFERRED_PROVIDER_virtual/libc-locale = glibc-locale
NOTE: selecting glibc-locale to satisfy runtime locale-base-el-gr due to
PREFERRED_PROVIDER_virtual/libc-locale = glibc-locale

But then insane.bbclass doesn't find rdep_data with "rdep_data =
oe.packagedata.read_subpkgdata(rdepend, d)", because there is locale-base-*
only for en-us in our build:

$ ls pkgdata/qemux86/runtime/locale-base-*
pkgdata/qemux86/runtime/locale-base-en-us
pkgdata/qemux86/runtime/locale-base-en-us.packaged

Which leads to:
GLIBC_GENERATE_LOCALES="en_US.UTF-8"
set in our builds.

I'm sorry for noise, I wish it would be more obvious, especially because
there was still one more mystery with bash recipe which has:
RDEPENDS_${PN}-ptest_append_libc-glibc = " locale-base-fr-fr
locale-base-de-de"
and doesn't trigger the QA errors in our builds, and I've discovered that
I've found about this long time ago and added work around in out bash
bbappend:

# We need to revert changes from
#
http://git.openembedded.org/openembedded-core/commit/?h=rocko&id=70e544452b6825686f06484d994936ded677825f
# because as described in:
#
http://lists.openembedded.org/pipermail/openembedded-commits/2017-November/215520.html
# it doesn't work when GLIBC_GENERATE_LOCALES are restricted like they are
in our builds:
#
meta-distro1/conf/distro/include/distro1-toolchain.inc:GLIBC_GENERATE_LOCALES
= "en_US.UTF-8"
#
meta-distro2/conf/distro/include/distro2-toolchain.inc:GLIBC_GENERATE_LOCALES
= "en_US.UTF-8"
RDEPENDS_${PN}-ptest_remove_libc-glibc = "locale-base-fr-fr
locale-base-de-de"

So clearly a year is too long for my memory to remember tweaks like this
especially when bitbake -e doesn't show them (more about that bellow).

Maybe we should improve the comment in meta/conf/local.conf.sample.extended
to explain possible issues with ptest when GLIBC_GENERATE_LOCALES doesn't
include certain locales? Now there is only this paragraph:

# Set GLIBC_GENERATE_LOCALES to the locales you wish to generate should you
not
# wish to perform the time-consuming step of generating all LIBC locales.
# NOTE: If removing en_US.UTF-8 you will also need to uncomment, and set
# appropriate value for IMAGE_LINGUAS.
# WARNING: this may break localisation!
#GLIBC_GENERATE_LOCALES = "en_GB.UTF-8 en_US.UTF-8"
#IMAGE_LINGUAS ?= "en-gb"

I'll send patch for that.

There is locale-base-fr-fr listed twice in glib.inc, I'll send patch for
that as well.

I wonder why locale-base isn't shown at all in bitbake -e bash with my
bbappend removing the dependency:

# $RDEPENDS_bash-ptest [3 operations]
#   rename from RDEPENDS_${PN}-ptest data.py:116 [expandKeys]
#     "make"
#   override[class-native]:rename from RDEPENDS_${PN}-ptest_class-native
data_smart.py:641 [renameVar]
#     ""
#   override[class-nativesdk]:rename from
RDEPENDS_${PN}-ptest_class-nativesdk data_smart.py:641 [renameVar]
#     ""
# pre-expansion value:
#   "make"
RDEPENDS_bash-ptest="make"

There isn't any history about RDEPENDS_${PN}-ptest before the rename.

$ grep RDEPENDS.*ptest env.bash
#   override[bash-ptest]:rename from RDEPENDS_${PN}-ptest data.py:116
[expandKeys]
#   override[bash-ptest_class-native]:rename from
RDEPENDS_${PN}-ptest_class-native data_smart.py:641 [renameVar]
#   override[bash-ptest_class-nativesdk]:rename from
RDEPENDS_${PN}-ptest_class-nativesdk data_smart.py:641 [renameVar]
# $RDEPENDS_bash-ptest [3 operations]
#   rename from RDEPENDS_${PN}-ptest data.py:116 [expandKeys]
#   override[class-native]:rename from RDEPENDS_${PN}-ptest_class-native
data_smart.py:641 [renameVar]
#   override[class-nativesdk]:rename from
RDEPENDS_${PN}-ptest_class-nativesdk data_smart.py:641 [renameVar]
RDEPENDS_bash-ptest="make"
# $RDEPENDS_bash-ptest_class-native
#   rename from RDEPENDS_${PN}-ptest_class-native data_smart.py:641
[renameVar]
RDEPENDS_bash-ptest_class-native=""
# $RDEPENDS_bash-ptest_class-nativesdk
#   rename from RDEPENDS_${PN}-ptest_class-nativesdk data_smart.py:641
[renameVar]
RDEPENDS_bash-ptest_class-nativesdk=""

$ grep locale-base env.bash
$

And in the build without the bbappend:

# $RDEPENDS_bash-ptest [3 operations]
#   rename from RDEPENDS_${PN}-ptest data.py:116 [expandKeys]
#     " make"
#   override[class-native]:rename from RDEPENDS_${PN}-ptest_class-native
data_smart.py:641 [renameVar]
#     ""
#   override[class-nativesdk]:rename from
RDEPENDS_${PN}-ptest_class-nativesdk data_smart.py:641 [renameVar]
#     ""
# pre-expansion value:
#   " make locale-base-fr-fr locale-base-de-de"
RDEPENDS_bash-ptest=" make locale-base-fr-fr locale-base-de-de"

but also without the RDEPENDS_${PN}-ptest history:

$ grep RDEPENDS.*ptest env.bash
#   override[bash-ptest]:rename from RDEPENDS_${PN}-ptest data.py:116
[expandKeys]
#   override[bash-ptest_class-native]:rename from
RDEPENDS_${PN}-ptest_class-native data_smart.py:641 [renameVar]
#   override[bash-ptest_class-nativesdk]:rename from
RDEPENDS_${PN}-ptest_class-nativesdk data_smart.py:641 [renameVar]
# $RDEPENDS_bash-ptest [3 operations]
#   rename from RDEPENDS_${PN}-ptest data.py:116 [expandKeys]
#   override[class-native]:rename from RDEPENDS_${PN}-ptest_class-native
data_smart.py:641 [renameVar]
#   override[class-nativesdk]:rename from
RDEPENDS_${PN}-ptest_class-nativesdk data_smart.py:641 [renameVar]
RDEPENDS_bash-ptest=" make locale-base-fr-fr locale-base-de-de"
# $RDEPENDS_bash-ptest_class-native
#   rename from RDEPENDS_${PN}-ptest_class-native data_smart.py:641
[renameVar]
RDEPENDS_bash-ptest_class-native=""
# $RDEPENDS_bash-ptest_class-nativesdk
#   rename from RDEPENDS_${PN}-ptest_class-nativesdk data_smart.py:641
[renameVar]
RDEPENDS_bash-ptest_class-nativesdk=""

$ grep locale-base env.bash
#   " make locale-base-fr-fr locale-base-de-de"
RDEPENDS_bash-ptest=" make locale-base-fr-fr locale-base-de-de"

The same with glib-2.0:
# $RDEPENDS_glib-2.0-ptest [3 operations]
#   rename from RDEPENDS_${PN}-ptest data.py:116 [expandKeys]
#     " gnome-desktop-testing             dbus
 gnome-desktop-testing             tzdata             tzdata-americas
       tzdata-asia             tzdata-europe             tzdata-posix
       shared-mime-info            "
#   override[class-native]:rename from RDEPENDS_${PN}-ptest_class-native
data_smart.py:641 [renameVar]
#     ""
#   override[class-nativesdk]:rename from
RDEPENDS_${PN}-ptest_class-nativesdk data_smart.py:641 [renameVar]
#     ""
# pre-expansion value:
#   " gnome-desktop-testing             dbus
 gnome-desktop-testing             tzdata             tzdata-americas
       tzdata-asia             tzdata-europe             tzdata-posix
       shared-mime-info                        glibc-gconv-utf-16
   glibc-charmap-utf-8             glibc-gconv-cp1255
 glibc-charmap-cp1255             glibc-gconv-utf-32
 glibc-gconv-utf-7             glibc-gconv-euc-jp
 glibc-gconv-iso8859-1             glibc-gconv-iso8859-15
 glibc-charmap-invariant
 glibc-localedata-translit-cjk-variants             locale-base-tr-tr
       locale-base-lt-lt             locale-base-ja-jp.euc-jp
 locale-base-fa-ir             locale-base-ru-ru
 locale-base-de-de             locale-base-hr-hr
 locale-base-el-gr             locale-base-fr-fr
 locale-base-fr-fr             locale-base-es-es
 locale-base-en-gb            "
RDEPENDS_glib-2.0-ptest=" gnome-desktop-testing             dbus
 gnome-desktop-testing             tzdata             tzdata-americas
       tzdata-asia             tzdata-europe             tzdata-posix
       shared-mime-info                        glibc-gconv-utf-16
   glibc-charmap-utf-8             glibc-gconv-cp1255
 glibc-charmap-cp1255             glibc-gconv-utf-32
 glibc-gconv-utf-7             glibc-gconv-euc-jp
 glibc-gconv-iso8859-1             glibc-gconv-iso8859-15
 glibc-charmap-invariant
 glibc-localedata-translit-cjk-variants             locale-base-tr-tr
       locale-base-lt-lt             locale-base-ja-jp.euc-jp
 locale-base-fa-ir             locale-base-ru-ru
 locale-base-de-de             locale-base-hr-hr
 locale-base-el-gr             locale-base-fr-fr
 locale-base-fr-fr             locale-base-es-es
 locale-base-en-gb            "

It just magically appears in pre-expansion value.

Shouldn't bitbake -e show that "locale-base-fr-fr locale-base-de-de" were
appended (and in the build with bbappend that they were appended and then
removed)? That would help me to find out what's going on much faster.

Cheers,





On Sat, Oct 13, 2018 at 12:29 PM Richard Purdie <
richard.purdie@linuxfoundation.org> wrote:

> On Sat, 2018-10-13 at 02:15 +0200, Martin Jansa wrote:
> > Hi,
> >
> > sorry for delay, I was first testing a bit more specific fix:
> > DEPENDS_append_libc-glibc = "${@bb.utils.contains('PTEST_ENABLED',
> > '1', '', ' virtual/libc-locale', d)}"
> >
> > but unfortunately this nor your shorter version work.
> >
> > and it's reproducible without the multilib enabled as well:
> >
> > ERROR: glib-2.0-1_2.58.0-r0 do_package_qa: QA Issue: glib-2.0-ptest
> rdepends on locale-base-de-de, but it isn't a build dependency? [build-deps]
> > ERROR: glib-2.0-1_2.58.0-r0 do_package_qa: QA Issue: glib-2.0-ptest
> rdepends on locale-base-el-gr, but it isn't a build dependency? [build-deps]
> > ERROR: glib-2.0-1_2.58.0-r0 do_package_qa: QA Issue: glib-2.0-ptest
> rdepends on locale-base-en-gb, but it isn't a build dependency? [build-deps]
>
> What is also puzzling me is that we're not seeing this in OE-Core on
> the autobuilders. We recently merged much better warning summary code
> so we should see this quite clearly in its output and we're not...
>
> I'm not sure what that means but it suggests there is something going
> on here we're not quite seeing...
>
> Cheers,
>
> Richard
>
>
>

[-- Attachment #2: Type: text/html, Size: 17052 bytes --]

Re: [oe-commits] [openembedded-core] 02/09: glib-2.0: add missing locale dependencies to PN-ptest

[richard.purdie]

On Sat, 2018-10-13 at 23:16 +0200, Martin Jansa wrote:
> I was checking my builds and here is what I've found.

Thanks for digging into this, it has highlighted a few important
issues. I've queued the patches.

> $ grep locale-base env.bash
> $
> 
> And in the build without the bbappend:
> 
> # $RDEPENDS_bash-ptest [3 operations]
> #   rename from RDEPENDS_${PN}-ptest data.py:116 [expandKeys]
> #     " make"
> #   override[class-native]:rename from RDEPENDS_${PN}-ptest_class-
> native data_smart.py:641 [renameVar]
> #     ""
> #   override[class-nativesdk]:rename from RDEPENDS_${PN}-ptest_class-
> nativesdk data_smart.py:641 [renameVar]
> #     ""
> # pre-expansion value:
> #   " make locale-base-fr-fr locale-base-de-de"
> RDEPENDS_bash-ptest=" make locale-base-fr-fr locale-base-de-de"
> 
> but also without the RDEPENDS_${PN}-ptest history:
> 
> $ grep RDEPENDS.*ptest env.bash
> #   override[bash-ptest]:rename from RDEPENDS_${PN}-ptest data.py:116
> [expandKeys]
> #   override[bash-ptest_class-native]:rename from RDEPENDS_${PN}-
> ptest_class-native data_smart.py:641 [renameVar]
> #   override[bash-ptest_class-nativesdk]:rename from RDEPENDS_${PN}-
> ptest_class-nativesdk data_smart.py:641 [renameVar]
> # $RDEPENDS_bash-ptest [3 operations]
> #   rename from RDEPENDS_${PN}-ptest data.py:116 [expandKeys]
> #   override[class-native]:rename from RDEPENDS_${PN}-ptest_class-
> native data_smart.py:641 [renameVar]
> #   override[class-nativesdk]:rename from RDEPENDS_${PN}-ptest_class-
> nativesdk data_smart.py:641 [renameVar]
> RDEPENDS_bash-ptest=" make locale-base-fr-fr locale-base-de-de"
> # $RDEPENDS_bash-ptest_class-native
> #   rename from RDEPENDS_${PN}-ptest_class-native data_smart.py:641
> [renameVar]
> RDEPENDS_bash-ptest_class-native=""
> # $RDEPENDS_bash-ptest_class-nativesdk
> #   rename from RDEPENDS_${PN}-ptest_class-nativesdk
> data_smart.py:641 [renameVar]
> RDEPENDS_bash-ptest_class-nativesdk=""
> 
> $ grep locale-base env.bash
> #   " make locale-base-fr-fr locale-base-de-de"
> RDEPENDS_bash-ptest=" make locale-base-fr-fr locale-base-de-de"
[...]
> It just magically appears in pre-expansion value.
> 
> Shouldn't bitbake -e show that "locale-base-fr-fr locale-base-de-de"
> were appended (and in the build with bbappend that they were appended
> and then removed)? That would help me to find out what's going on
> much faster.

Agreed, bitbake needs to do better here. I had a quick poke around and
came up with this:

diff --git a/bitbake/lib/bb/data_smart.py b/bitbake/lib/bb/data_smart.py
index 7b09af5cf1b..4434142a02e 100644
--- a/bitbake/lib/bb/data_smart.py
+++ b/bitbake/lib/bb/data_smart.py
@@ -267,6 +267,16 @@ class VariableHistory(object):
             return
         self.variables[var].append(loginfo.copy())
 
+    def rename_variable_hist(self, oldvar, newvar):
+        if not self.dataroot._tracking:
+            return
+        if oldvar not in self.variables:
+            return
+        if newvar not in self.variables:
+            self.variables[newvar] = []
+        for i in self.variables[oldvar]:
+            self.variables[newvar].append(i.copy())
+
     def variable(self, var):
         remote_connector = self.dataroot.getVar('_remote_data', False)
         if remote_connector:
@@ -619,6 +629,7 @@ class DataSmart(MutableMapping):
 
         val = self.getVar(key, 0, parsing=True)
         if val is not None:
+            self.varhistory.rename_variable_hist(key, newkey)
             loginfo['variable'] = newkey
             loginfo['op'] = 'rename from %s' % key
             loginfo['detail'] = val


which now shows:

# $RDEPENDS_bash-ptest [8 operations]
#   append /media/build1/poky/meta/recipes-extended/bash/bash.inc:26
#     "make"
#   _append[libc-glibc] /media/build1/poky/meta/recipes-extended/bash/bash.inc:29
#     " locale-base-fr-fr locale-base-de-de"
#   _remove[libc-glibc] /media/build1/poky/meta/recipes-extended/bash/bash_4.4.18.bb:43
#     "locale-base-fr-fr locale-base-de-de"
#   rename from RDEPENDS_${PN}-ptest data.py:116 [expandKeys]
#     "make"
#   override[class-native]:set /media/build1/poky/meta/classes/ptest.bbclass:13
#     ""
#   override[class-native]:rename from RDEPENDS_${PN}-ptest_class-native data_smart.py:652 [renameVar]
#     ""
#   override[class-nativesdk]:set /media/build1/poky/meta/classes/ptest.bbclass:14
#     ""
#   override[class-nativesdk]:rename from RDEPENDS_${PN}-ptest_class-nativesdk data_smart.py:652 [renameVar]
#     ""
# pre-expansion value:
#   "make"
RDEPENDS_bash-ptest="make"

which is better but I'm confused why there are the "rename from" for
the overrides which aren't being applied (this is for bash, not bash-
native or bash-nativesdk).

So more investigation is needed...

Cheers,

Richard

Re: [oe-commits] [openembedded-core] 02/09: glib-2.0: add missing locale dependencies to PN-ptest

[Alexander Kanavin]

This is probably related to this issue?
https://bugzilla.yoctoproject.org/show_bug.cgi?id=12913

Alex
On Sun, 14 Oct 2018 at 13:21, <richard.purdie@linuxfoundation.org> wrote:
>
> On Sat, 2018-10-13 at 23:16 +0200, Martin Jansa wrote:
> > I was checking my builds and here is what I've found.
>
> Thanks for digging into this, it has highlighted a few important
> issues. I've queued the patches.
>
> > $ grep locale-base env.bash
> > $
> >
> > And in the build without the bbappend:
> >
> > # $RDEPENDS_bash-ptest [3 operations]
> > #   rename from RDEPENDS_${PN}-ptest data.py:116 [expandKeys]
> > #     " make"
> > #   override[class-native]:rename from RDEPENDS_${PN}-ptest_class-
> > native data_smart.py:641 [renameVar]
> > #     ""
> > #   override[class-nativesdk]:rename from RDEPENDS_${PN}-ptest_class-
> > nativesdk data_smart.py:641 [renameVar]
> > #     ""
> > # pre-expansion value:
> > #   " make locale-base-fr-fr locale-base-de-de"
> > RDEPENDS_bash-ptest=" make locale-base-fr-fr locale-base-de-de"
> >
> > but also without the RDEPENDS_${PN}-ptest history:
> >
> > $ grep RDEPENDS.*ptest env.bash
> > #   override[bash-ptest]:rename from RDEPENDS_${PN}-ptest data.py:116
> > [expandKeys]
> > #   override[bash-ptest_class-native]:rename from RDEPENDS_${PN}-
> > ptest_class-native data_smart.py:641 [renameVar]
> > #   override[bash-ptest_class-nativesdk]:rename from RDEPENDS_${PN}-
> > ptest_class-nativesdk data_smart.py:641 [renameVar]
> > # $RDEPENDS_bash-ptest [3 operations]
> > #   rename from RDEPENDS_${PN}-ptest data.py:116 [expandKeys]
> > #   override[class-native]:rename from RDEPENDS_${PN}-ptest_class-
> > native data_smart.py:641 [renameVar]
> > #   override[class-nativesdk]:rename from RDEPENDS_${PN}-ptest_class-
> > nativesdk data_smart.py:641 [renameVar]
> > RDEPENDS_bash-ptest=" make locale-base-fr-fr locale-base-de-de"
> > # $RDEPENDS_bash-ptest_class-native
> > #   rename from RDEPENDS_${PN}-ptest_class-native data_smart.py:641
> > [renameVar]
> > RDEPENDS_bash-ptest_class-native=""
> > # $RDEPENDS_bash-ptest_class-nativesdk
> > #   rename from RDEPENDS_${PN}-ptest_class-nativesdk
> > data_smart.py:641 [renameVar]
> > RDEPENDS_bash-ptest_class-nativesdk=""
> >
> > $ grep locale-base env.bash
> > #   " make locale-base-fr-fr locale-base-de-de"
> > RDEPENDS_bash-ptest=" make locale-base-fr-fr locale-base-de-de"
> [...]
> > It just magically appears in pre-expansion value.
> >
> > Shouldn't bitbake -e show that "locale-base-fr-fr locale-base-de-de"
> > were appended (and in the build with bbappend that they were appended
> > and then removed)? That would help me to find out what's going on
> > much faster.
>
> Agreed, bitbake needs to do better here. I had a quick poke around and
> came up with this:
>
> diff --git a/bitbake/lib/bb/data_smart.py b/bitbake/lib/bb/data_smart.py
> index 7b09af5cf1b..4434142a02e 100644
> --- a/bitbake/lib/bb/data_smart.py
> +++ b/bitbake/lib/bb/data_smart.py
> @@ -267,6 +267,16 @@ class VariableHistory(object):
>              return
>          self.variables[var].append(loginfo.copy())
>
> +    def rename_variable_hist(self, oldvar, newvar):
> +        if not self.dataroot._tracking:
> +            return
> +        if oldvar not in self.variables:
> +            return
> +        if newvar not in self.variables:
> +            self.variables[newvar] = []
> +        for i in self.variables[oldvar]:
> +            self.variables[newvar].append(i.copy())
> +
>      def variable(self, var):
>          remote_connector = self.dataroot.getVar('_remote_data', False)
>          if remote_connector:
> @@ -619,6 +629,7 @@ class DataSmart(MutableMapping):
>
>          val = self.getVar(key, 0, parsing=True)
>          if val is not None:
> +            self.varhistory.rename_variable_hist(key, newkey)
>              loginfo['variable'] = newkey
>              loginfo['op'] = 'rename from %s' % key
>              loginfo['detail'] = val
>
>
> which now shows:
>
> # $RDEPENDS_bash-ptest [8 operations]
> #   append /media/build1/poky/meta/recipes-extended/bash/bash.inc:26
> #     "make"
> #   _append[libc-glibc] /media/build1/poky/meta/recipes-extended/bash/bash.inc:29
> #     " locale-base-fr-fr locale-base-de-de"
> #   _remove[libc-glibc] /media/build1/poky/meta/recipes-extended/bash/bash_4.4.18.bb:43
> #     "locale-base-fr-fr locale-base-de-de"
> #   rename from RDEPENDS_${PN}-ptest data.py:116 [expandKeys]
> #     "make"
> #   override[class-native]:set /media/build1/poky/meta/classes/ptest.bbclass:13
> #     ""
> #   override[class-native]:rename from RDEPENDS_${PN}-ptest_class-native data_smart.py:652 [renameVar]
> #     ""
> #   override[class-nativesdk]:set /media/build1/poky/meta/classes/ptest.bbclass:14
> #     ""
> #   override[class-nativesdk]:rename from RDEPENDS_${PN}-ptest_class-nativesdk data_smart.py:652 [renameVar]
> #     ""
> # pre-expansion value:
> #   "make"
> RDEPENDS_bash-ptest="make"
>
> which is better but I'm confused why there are the "rename from" for
> the overrides which aren't being applied (this is for bash, not bash-
> native or bash-nativesdk).
>
> So more investigation is needed...
>
> Cheers,
>
> Richard
>
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [oe-commits] [openembedded-core] 02/09: glib-2.0: add missing locale dependencies to PN-ptest

[richard.purdie]

On Sun, 2018-10-14 at 13:40 +0200, Alexander Kanavin wrote:
> This is probably related to this issue?
> https://bugzilla.yoctoproject.org/show_bug.cgi?id=12913

I don't think it is, no, other than both involving the remove
operator...

Cheers,

Richard