[RFC v1 PATCH 00/16] populate perl-native into its own directory

[RFC v1 PATCH 00/16] populate perl-native into its own directory

[Dexuan Cui]

Currently both perl-native (a.k.a. ${STAGING_BINDIR_NATIVE}/perl )and
perl-native-runtime(a.k.a. the system perl, or /usr/bin/perl) appear in
the PATH when bitbake is running.
This can cause some race conditions: many places detecting perl from PATH
can't make sure which path will be used as this depends on when perl-native's
populate_sysroot is finished, e.g., automake-native and autoconf-native could
use perl-native-runtime while gnu-config-native could use perl-native and
this inconsistent usages can cause trouble, e.g., bug 941.

And, as RP suggested, "the time to use perl-native instead of
perl-native-runtime is where any perl module is being built, perl itself is
being built or anything that has an explicit dependency on the perl version
present".

So I made the following changes to try to address the aboves issues:
1) populate perl-native into its own directory so it won't appear in PATH
by default, and add perlnative.bbclass for these recipes that really depend
on perl-native;
2) check all perl-native references and correct the ones that should be
perl-native-runtime;
3) fix any building issues due to the new location of perl-native,
especially cpan and cpan-base .bbclass.

With the changes, bug 941 doesn't appear.

Tests I did are:
I tried "bitbale core-image-sato-sdk and meta-toolchain-gmae" in x86_32 and
x86_64 Ubuntu hosts and everything seems building fine.


Please review the changes and comment on them. Thanks!

--------------------------

The following changes since commit 0736b136efac283449c9218c01740eba4a7bc8cc:

are available in the git repository at:
  git://git.pokylinux.org/poky-contrib dcui/master_perl-native
  http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=dcui/master_perl-native

Dexuan Cui (16):
  native.bbclass: allow a native package to be populated into its own
    dir
  perl-native: populate into its own dir
  perlnative.bbclass: add the file
  gnu-config-native: should depend on perl-native-runtime rather than
    perl-native
  libcap: should depend on perl-native-runtime rather than perl-native
  openssl: should depend on perl-native-runtime rather than perl-native
  git: should depend on perl-native-runtime rather than perl-native
  coreutils: remove unnecessary dependency on perl
  dpkg: should depend on perl-native-runtime rather than perl-native
  webkit-gtk: should depend on perl-native-runtime rather than
    perl-native
  perl: inherit perlnative
  cpan.bbclass, cpan-base.bbclas: update them for the perlnative change
  libxml-parser-perl: inherit perlnative
  libconvert-asn1-perl: fix EXTRA_PERLFLAGS due to the perl-native
    change
  libxml-simple-perl: fix EXTRA_PERLFLAGS due the the perlnative change
  icon-naming-utils-native: inherit perlnative

 meta/classes/cpan-base.bbclass                     |   10 +++++++---
 meta/classes/cpan.bbclass                          |   12 ++++++------
 meta/classes/native.bbclass                        |    5 +++++
 meta/classes/perlnative.bbclass                    |    2 ++
 meta/recipes-connectivity/openssl/openssl.inc      |    2 +-
 .../recipes-connectivity/openssl/openssl_0.9.8r.bb |    2 +-
 meta/recipes-core/coreutils/coreutils_6.9.bb       |    6 +++---
 meta/recipes-core/coreutils/coreutils_8.9.bb       |    6 +++---
 meta/recipes-devtools/dpkg/dpkg.inc                |    2 +-
 meta/recipes-devtools/dpkg/dpkg_1.15.8.7.bb        |    2 +-
 meta/recipes-devtools/git/git.inc                  |    4 ++--
 meta/recipes-devtools/git/git_1.7.5.1.bb           |    2 +-
 .../gnu-config/gnu-config_20080123.bb              |    4 ++--
 .../icon-naming-utils-native_0.8.7.bb              |    4 ++--
 .../perl/libxml-parser-perl-native_2.40.bb         |    2 +-
 .../perl/libxml-parser-perl_2.40.bb                |    2 +-
 .../perl/libxml-simple-perl-native_2.18.bb         |    2 +-
 .../perl/libxml-simple-perl_2.18.bb                |    4 ++--
 meta/recipes-devtools/perl/perl-native_5.12.3.bb   |    5 ++++-
 meta/recipes-devtools/perl/perl_5.12.3.bb          |   10 +++++-----
 .../perl/libconvert-asn1-perl_0.22.bb              |    4 ++--
 meta/recipes-sato/webkit/webkit-gtk_svn.bb         |    6 +++---
 meta/recipes-support/libcap/libcap.inc             |    4 ++--
 meta/recipes-support/libcap/libcap_2.20.bb         |    2 +-
 24 files changed, 59 insertions(+), 45 deletions(-)
 create mode 100644 meta/classes/perlnative.bbclass

-- 
1.7.2

[RFC v1 PATCH 01/16] native.bbclass: allow a native package to be populated into its own dir

[Dexuan Cui]

Signed-off-by: Dexuan Cui <dexuan.cui@intel.com>
---
 meta/classes/native.bbclass |    5 +++++
 1 files changed, 5 insertions(+), 0 deletions(-)

diff --git a/meta/classes/native.bbclass b/meta/classes/native.bbclass
index e06f02a..8a3c294 100644
--- a/meta/classes/native.bbclass
+++ b/meta/classes/native.bbclass
@@ -67,6 +67,11 @@ base_prefix = "${STAGING_DIR_NATIVE}"
 prefix = "${STAGING_DIR_NATIVE}${prefix_native}"
 exec_prefix = "${STAGING_DIR_NATIVE}${prefix_native}"
 
+PACKAGE_OWN_DIR = ""
+bindir .= "${PACKAGE_OWN_DIR}"
+libdir .= "${PACKAGE_OWN_DIR}"
+libexecdir .= "${PACKAGE_OWN_DIR}"
+
 do_populate_sysroot[sstate-inputdirs] = "${SYSROOT_DESTDIR}/${STAGING_DIR_NATIVE}"
 do_populate_sysroot[sstate-outputdirs] = "${STAGING_DIR_NATIVE}"
 
-- 
1.7.2

Re: [RFC v1 PATCH 01/16] native.bbclass: allow a native package to be populated into its own dir

[Phil Blundell]

On Wed, 2011-06-01 at 21:18 +0800, Dexuan Cui wrote:
> +PACKAGE_OWN_DIR = ""
> +bindir .= "${PACKAGE_OWN_DIR}"
> +libdir .= "${PACKAGE_OWN_DIR}"
> +libexecdir .= "${PACKAGE_OWN_DIR}"

I think the general idea of this patch is a good one but I'm not
terribly fond of the name of that variable.  If this is meant to be a
path suffix that you can optionally set for native packages then how
about calling it something like NATIVE_PACKAGE_PATH_SUFFIX?

p.

Re: [RFC v1 PATCH 01/16] native.bbclass: allow a native package to be populated into its own dir

[Cui, Dexuan]

Phil Blundell wrote:
> On Wed, 2011-06-01 at 21:18 +0800, Dexuan Cui wrote:
>> +PACKAGE_OWN_DIR = ""
>> +bindir .= "${PACKAGE_OWN_DIR}"
>> +libdir .= "${PACKAGE_OWN_DIR}"
>> +libexecdir .= "${PACKAGE_OWN_DIR}"
> 
> I think the general idea of this patch is a good one but I'm not
> terribly fond of the name of that variable.  If this is meant to be a
> path suffix that you can optionally set for native packages then how
> about calling it something like NATIVE_PACKAGE_PATH_SUFFIX?

Hi Phil,
Thanks a lot for the suggestion.
Surely NATIVE_PACKAGE_PATH_SUFFIX is a much better naming! :-)
I'll change to use it.

Thanks,
-- Dexuan

Re: [RFC v1 PATCH 01/16] native.bbclass: allow a native package to be populated into its own dir

[Cui, Dexuan]

Cui, Dexuan wrote:
> Phil Blundell wrote:
>> On Wed, 2011-06-01 at 21:18 +0800, Dexuan Cui wrote:
>>> +PACKAGE_OWN_DIR = ""
>>> +bindir .= "${PACKAGE_OWN_DIR}"
>>> +libdir .= "${PACKAGE_OWN_DIR}"
>>> +libexecdir .= "${PACKAGE_OWN_DIR}"
>> 
>> I think the general idea of this patch is a good one but I'm not
>> terribly fond of the name of that variable.  If this is meant to be a
>> path suffix that you can optionally set for native packages then how
>> about calling it something like NATIVE_PACKAGE_PATH_SUFFIX?
> 
> Hi Phil,
> Thanks a lot for the suggestion.
> Surely NATIVE_PACKAGE_PATH_SUFFIX is a much better naming! :-)
> I'll change to use it.

Hi RP, Phil, 
A new branch dcui/master_perl-native_v2 was created for this better naming(this is the only difference compared with the previous version):
http://git.pokylinux.org/cgit/cgit.cgi/poky-contrib/log/?h=dcui/master_perl-native_v2

Thanks,
-- Dexuan

[RFC v1 PATCH 02/16] perl-native: populate into its own dir

[Dexuan Cui]

Signed-off-by: Dexuan Cui <dexuan.cui@intel.com>
---
 meta/recipes-devtools/perl/perl-native_5.12.3.bb |    5 ++++-
 1 files changed, 4 insertions(+), 1 deletions(-)

diff --git a/meta/recipes-devtools/perl/perl-native_5.12.3.bb b/meta/recipes-devtools/perl/perl-native_5.12.3.bb
index cbb4e78..56274f0 100644
--- a/meta/recipes-devtools/perl/perl-native_5.12.3.bb
+++ b/meta/recipes-devtools/perl/perl-native_5.12.3.bb
@@ -4,7 +4,7 @@ SECTION = "libs"
 LICENSE = "Artistic|GPL"
 LIC_FILES_CHKSUM = "file://Copying;md5=2b4c6ffbcfcbdee469f02565f253d81a \
 		    file://Artistic;md5=f921793d03cc6d63ec4b15e9be8fd3f8"
-PR = "r2"
+PR = "r3"
 
 LIC_FILES_CHKSUM = "file://Copying;md5=2b4c6ffbcfcbdee469f02565f253d81a \
                     file://Artistic;md5=f921793d03cc6d63ec4b15e9be8fd3f8"
@@ -28,6 +28,8 @@ S = "${WORKDIR}/perl-${PV}"
 
 inherit native
 
+PACKAGE_OWN_DIR = "/${PN}"
+
 export LD="${CCLD}"
 
 do_configure () {
@@ -41,6 +43,7 @@ do_configure () {
 		-Dvendorprefix=${prefix} \
 		-Dsiteprefix=${prefix} \
 		\
+		-Dbin=${STAGING_BINDIR}/${PN} \
 		-Dprivlib=${STAGING_LIBDIR}/perl/${PV} \
 		-Darchlib=${STAGING_LIBDIR}/perl/${PV} \
 		-Dvendorlib=${STAGING_LIBDIR}/perl/${PV} \
-- 
1.7.2

[RFC v1 PATCH 03/16] perlnative.bbclass: add the file

[Dexuan Cui]

Signed-off-by: Dexuan Cui <dexuan.cui@intel.com>
---
 meta/classes/perlnative.bbclass |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)
 create mode 100644 meta/classes/perlnative.bbclass

diff --git a/meta/classes/perlnative.bbclass b/meta/classes/perlnative.bbclass
new file mode 100644
index 0000000..522344d
--- /dev/null
+++ b/meta/classes/perlnative.bbclass
@@ -0,0 +1,2 @@
+PATH_prepend = "${STAGING_BINDIR_NATIVE}/perl-native:"
+DEPENDS += "perl-native"
-- 
1.7.2

[RFC v1 PATCH 04/16] gnu-config-native: should depend on perl-native-runtime rather than perl-native

[Dexuan Cui]

Signed-off-by: Dexuan Cui <dexuan.cui@intel.com>
---
 .../gnu-config/gnu-config_20080123.bb              |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/meta/recipes-devtools/gnu-config/gnu-config_20080123.bb b/meta/recipes-devtools/gnu-config/gnu-config_20080123.bb
index 897984d..de07f43 100644
--- a/meta/recipes-devtools/gnu-config/gnu-config_20080123.bb
+++ b/meta/recipes-devtools/gnu-config/gnu-config_20080123.bb
@@ -4,13 +4,13 @@ SECTION = "devel"
 LICENSE = "GPLv1+"
 LIC_FILES_CHKSUM = "file://config.guess;endline=39;md5=a089987af4a25cb0419d1c2fd6d495e3"
 
-DEPENDS_virtclass-native = "perl-native"
+DEPENDS_virtclass-native = "perl-native-runtime"
 
 INHIBIT_DEFAULT_DEPS = "1"
 
 FIXEDSRCDATE = "${@bb.data.getVar('FILE', d, 1).split('_')[-1].split('.')[0]}"
 PV = "0.1+cvs${FIXEDSRCDATE}"
-PR = "r3"
+PR = "r4"
 
 SRC_URI = "cvs://anonymous@cvs.sv.gnu.org/cvsroot/config;module=config;method=pserver;date=${FIXEDSRCDATE} \
 	   file://config-guess-uclibc.patch \
-- 
1.7.2

[RFC v1 PATCH 05/16] libcap: should depend on perl-native-runtime rather than perl-native

[Dexuan Cui]

Signed-off-by: Dexuan Cui <dexuan.cui@intel.com>
---
 meta/recipes-support/libcap/libcap.inc     |    4 ++--
 meta/recipes-support/libcap/libcap_2.20.bb |    2 +-
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/meta/recipes-support/libcap/libcap.inc b/meta/recipes-support/libcap/libcap.inc
index 1b4c0bc..93bdf95 100644
--- a/meta/recipes-support/libcap/libcap.inc
+++ b/meta/recipes-support/libcap/libcap.inc
@@ -5,9 +5,9 @@ HOMEPAGE = "http://sites.google.com/site/fullycapable/"
 LICENSE = "BSD | GPL"
 LIC_FILES_CHKSUM = "file://License;md5=731de803c1ccbcb05a9b3523279c8d7f"
 
-DEPENDS = "libpam attr perl-native"
+DEPENDS = "libpam attr perl-native-runtime"
 # attr and pam are disabled by EXTRA_OEMAKE_virtclass-native
-DEPENDS_virtclass-native = "perl-native"
+DEPENDS_virtclass-native = "perl-native-runtime"
 
 SRC_URI = "${KERNELORG_MIRROR}/linux/libs/security/linux-privs/libcap2/${BPN}-${PV}.tar.bz2"
 
diff --git a/meta/recipes-support/libcap/libcap_2.20.bb b/meta/recipes-support/libcap/libcap_2.20.bb
index 2fa949b..ab3638a 100644
--- a/meta/recipes-support/libcap/libcap_2.20.bb
+++ b/meta/recipes-support/libcap/libcap_2.20.bb
@@ -1,6 +1,6 @@
 require libcap.inc
 
-PR = "r0"
+PR = "r1"
 
 SRC_URI[md5sum] = "10e47ed32ca2214eb0e58780282d27b4"
 SRC_URI[sha256sum] = "20e7c1ea4d3d5c410efb3a6ff138dc417912fae316d883460dcd58d9803a9220"
-- 
1.7.2

[RFC v1 PATCH 06/16] openssl: should depend on perl-native-runtime rather than perl-native

[Dexuan Cui]

Signed-off-by: Dexuan Cui <dexuan.cui@intel.com>
---
 meta/recipes-connectivity/openssl/openssl.inc      |    2 +-
 .../recipes-connectivity/openssl/openssl_0.9.8r.bb |    2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/meta/recipes-connectivity/openssl/openssl.inc b/meta/recipes-connectivity/openssl/openssl.inc
index f1ac718..a338f0e 100644
--- a/meta/recipes-connectivity/openssl/openssl.inc
+++ b/meta/recipes-connectivity/openssl/openssl.inc
@@ -8,7 +8,7 @@ SECTION = "libs/network"
 LICENSE = "openssl"
 LIC_FILES_CHKSUM = "file://LICENSE;md5=f9a8f968107345e0b75aa8c2ecaa7ec8"
 
-DEPENDS = "perl-native"
+DEPENDS = "perl-native-runtime"
 
 SRC_URI = "http://www.openssl.org/source/openssl-${PV}.tar.gz \
            file://parallel-make-fix.patch \
diff --git a/meta/recipes-connectivity/openssl/openssl_0.9.8r.bb b/meta/recipes-connectivity/openssl/openssl_0.9.8r.bb
index 5fc38a3..48ec995 100644
--- a/meta/recipes-connectivity/openssl/openssl_0.9.8r.bb
+++ b/meta/recipes-connectivity/openssl/openssl_0.9.8r.bb
@@ -1,6 +1,6 @@
 require openssl.inc
 
-PR = "r0"
+PR = "r1"
 SRC_URI += "file://debian/ca.patch \
             file://debian/config-hurd.patch;apply=no \
             file://debian/debian-targets.patch \
-- 
1.7.2

[RFC v1 PATCH 07/16] git: should depend on perl-native-runtime rather than perl-native

[Dexuan Cui]

Signed-off-by: Dexuan Cui <dexuan.cui@intel.com>
---
 meta/recipes-devtools/git/git.inc        |    4 ++--
 meta/recipes-devtools/git/git_1.7.5.1.bb |    2 +-
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/meta/recipes-devtools/git/git.inc b/meta/recipes-devtools/git/git.inc
index c884f9c..3cdd1fa 100644
--- a/meta/recipes-devtools/git/git.inc
+++ b/meta/recipes-devtools/git/git.inc
@@ -1,14 +1,14 @@
 DESCRIPTION = "The git revision control system used by the Linux kernel developers"
 SECTION = "console/utils"
 LICENSE = "GPLv2"
-DEPENDS = "perl-native openssl curl zlib expat"
+DEPENDS = "perl-native-runtime openssl curl zlib expat"
 
 SRC_URI = "${KERNELORG_MIRROR}/software/scm/git/git-${PV}.tar.bz2 "
 S = "${WORKDIR}/git-${PV}"
 
 LIC_FILES_CHKSUM = "file://COPYING;md5=7c0d7ef03a7eb04ce795b0f60e68e7e1"
 
-EXTRA_OECONF = "--with-perl=${STAGING_BINDIR_NATIVE}/perl --without-tcltk"
+EXTRA_OECONF = "--without-tcltk"
 
 inherit autotools
 
diff --git a/meta/recipes-devtools/git/git_1.7.5.1.bb b/meta/recipes-devtools/git/git_1.7.5.1.bb
index bfdbf62..9aa2bdf 100644
--- a/meta/recipes-devtools/git/git_1.7.5.1.bb
+++ b/meta/recipes-devtools/git/git_1.7.5.1.bb
@@ -1,6 +1,6 @@
 require git.inc
 
-PR = "r0"
+PR = "r1"
 
 EXTRA_OECONF += "ac_cv_snprintf_returns_bogus=no ac_cv_c_c99_format=yes \
                  ac_cv_fread_reads_directories=${ac_cv_fread_reads_directories=yes} \
-- 
1.7.2

Re: [RFC v1 PATCH 07/16] git: should depend on perl-native-runtime rather than perl-native

[Joshua Lock]

On Wed, 2011-06-01 at 21:18 +0800, Dexuan Cui wrote:
> Signed-off-by: Dexuan Cui <dexuan.cui@intel.com>
> ---
>  meta/recipes-devtools/git/git.inc        |    4 ++--
>  meta/recipes-devtools/git/git_1.7.5.1.bb |    2 +-
>  2 files changed, 3 insertions(+), 3 deletions(-)
> 
> diff --git a/meta/recipes-devtools/git/git.inc b/meta/recipes-devtools/git/git.inc
> index c884f9c..3cdd1fa 100644
> --- a/meta/recipes-devtools/git/git.inc
> +++ b/meta/recipes-devtools/git/git.inc
> @@ -1,14 +1,14 @@
>  DESCRIPTION = "The git revision control system used by the Linux kernel developers"
>  SECTION = "console/utils"
>  LICENSE = "GPLv2"
> -DEPENDS = "perl-native openssl curl zlib expat"
> +DEPENDS = "perl-native-runtime openssl curl zlib expat"

This change made git-native fail to build for me until I installed
perl-ExtUtils-MakeMaker on my Fedora 15 workstation.

If this is expected we need to be documenting this additional host
dependency.

Tracking bug: http://bugzilla.yoctoproject.org/show_bug.cgi?id=1155

Regards,
Joshua
-- 
Joshua Lock
        Yocto Build System Monkey
        Intel Open Source Technology Centre

Re: [RFC v1 PATCH 07/16] git: should depend on perl-native-runtime rather than perl-native

[Cui, Dexuan]

Joshua Lock wrote:
> On Wed, 2011-06-01 at 21:18 +0800, Dexuan Cui wrote:
>> Signed-off-by: Dexuan Cui <dexuan.cui@intel.com>
>> ---
>>  meta/recipes-devtools/git/git.inc        |    4 ++--
>>  meta/recipes-devtools/git/git_1.7.5.1.bb |    2 +-
>>  2 files changed, 3 insertions(+), 3 deletions(-)
>> 
>> diff --git a/meta/recipes-devtools/git/git.inc
>> b/meta/recipes-devtools/git/git.inc index c884f9c..3cdd1fa 100644
>> --- a/meta/recipes-devtools/git/git.inc +++
>> b/meta/recipes-devtools/git/git.inc @@ -1,14 +1,14 @@
>>  DESCRIPTION = "The git revision control system used by the Linux
>>  kernel developers"  SECTION = "console/utils" LICENSE = "GPLv2"
>> -DEPENDS = "perl-native openssl curl zlib expat"
>> +DEPENDS = "perl-native-runtime openssl curl zlib expat"
> 
> This change made git-native fail to build for me until I installed
> perl-ExtUtils-MakeMaker on my Fedora 15 workstation.
> 
> If this is expected we need to be documenting this additional host
> dependency.
> 
> Tracking bug: http://bugzilla.yoctoproject.org/show_bug.cgi?id=1155

Hi Joshua,
Thanks for reporting this!

Cc-ed Richard:
It might be inappropriate to "add perl-ExtUtils-MakeMaker to the host package requirements"?
Looks git should depend on perl-native?

I've made a patch http://git.pokylinux.org/cgit/cgit.cgi/poky-contrib/commit/?h=dcui/distro&id=79356a51507f1872199ba38262b749277633b4d4 and I'll send the pull request.
Please review it.

Thanks,
-- Dexuan

[RFC v1 PATCH 08/16] coreutils: remove unnecessary dependency on perl

[Dexuan Cui]

OE's coreutils doesn't depend on perl, either.

Signed-off-by: Dexuan Cui <dexuan.cui@intel.com>
---
 meta/recipes-core/coreutils/coreutils_6.9.bb |    6 +++---
 meta/recipes-core/coreutils/coreutils_8.9.bb |    6 +++---
 2 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/meta/recipes-core/coreutils/coreutils_6.9.bb b/meta/recipes-core/coreutils/coreutils_6.9.bb
index f837a22..c9aea38 100644
--- a/meta/recipes-core/coreutils/coreutils_6.9.bb
+++ b/meta/recipes-core/coreutils/coreutils_6.9.bb
@@ -8,9 +8,9 @@ BUGTRACKER = "http://debbugs.gnu.org/coreutils"
 LICENSE = "GPLv2+"
 LIC_FILES_CHKSUM = "file://COPYING;md5=751419260aa954499f7abaabaa882bbe \
                     file://src/ls.c;startline=4;endline=16;md5=482a96d4f25010a4e13f8743e0c3685e"
-PR = "r1"
-DEPENDS = "perl-native coreutils-native-${PV}"
-DEPENDS_virtclass-native = "perl-native gettext-native"
+PR = "r2"
+DEPENDS = "coreutils-native-${PV}"
+DEPENDS_virtclass-native = "gettext-native"
 
 inherit autotools gettext
 
diff --git a/meta/recipes-core/coreutils/coreutils_8.9.bb b/meta/recipes-core/coreutils/coreutils_8.9.bb
index 955aaf5..7f96c6e 100644
--- a/meta/recipes-core/coreutils/coreutils_8.9.bb
+++ b/meta/recipes-core/coreutils/coreutils_8.9.bb
@@ -7,9 +7,9 @@ BUGTRACKER = "http://debbugs.gnu.org/coreutils"
 LICENSE = "GPLv3+"
 LIC_FILES_CHKSUM = "file://COPYING;md5=d32239bcb673463ab874e80d47fae504\
                     file://src/ls.c;startline=5;endline=16;md5=e1a509558876db58fb6667ba140137ad"
-PR = "r1"
-DEPENDS = "perl-native gmp"
-DEPENDS_virtclass-native = "perl-native"
+PR = "r2"
+DEPENDS = "gmp"
+DEPENDS_virtclass-native = ""
 
 inherit autotools gettext
 
-- 
1.7.2

[RFC v1 PATCH 09/16] dpkg: should depend on perl-native-runtime rather than perl-native

[Dexuan Cui]

Signed-off-by: Dexuan Cui <dexuan.cui@intel.com>
---
 meta/recipes-devtools/dpkg/dpkg.inc         |    2 +-
 meta/recipes-devtools/dpkg/dpkg_1.15.8.7.bb |    2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/meta/recipes-devtools/dpkg/dpkg.inc b/meta/recipes-devtools/dpkg/dpkg.inc
index 0284442..c8ff182 100644
--- a/meta/recipes-devtools/dpkg/dpkg.inc
+++ b/meta/recipes-devtools/dpkg/dpkg.inc
@@ -6,7 +6,7 @@ SRC_URI = "${DEBIAN_MIRROR}/main/d/dpkg/dpkg_${PV}.tar.bz2 \
            file://ignore_extra_fields.patch;patch=1"
 
 DEPENDS = "zlib bzip2 perl"
-DEPENDS_virtclass-native = "bzip2-native zlib-native virtual/update-alternatives-native gettext-native perl-native"
+DEPENDS_virtclass-native = "bzip2-native zlib-native virtual/update-alternatives-native gettext-native perl-native-runtime"
 RDEPENDS_${PN} = "${VIRTUAL-RUNTIME_update-alternatives}"
 RDEPENDS_${PN}_virtclass-native = ""
 
diff --git a/meta/recipes-devtools/dpkg/dpkg_1.15.8.7.bb b/meta/recipes-devtools/dpkg/dpkg_1.15.8.7.bb
index 8bb441f..22edab8 100644
--- a/meta/recipes-devtools/dpkg/dpkg_1.15.8.7.bb
+++ b/meta/recipes-devtools/dpkg/dpkg_1.15.8.7.bb
@@ -8,7 +8,7 @@ SRC_URI += "file://noman.patch;patch=1 \
 SRC_URI[md5sum] = "d1731d4147c1ea3b537a4d094519a6dc"
 SRC_URI[sha256sum] = "1ec1376471b04717a4497e5d7a27cd545248c92116898ce0c53ced8ea94267b5"
 
-PR = "r2"
+PR = "r3"
 
 EXTRA_OECONF = "--without-static-progs \
 		--without-dselect \
-- 
1.7.2

[RFC v1 PATCH 10/16] webkit-gtk: should depend on perl-native-runtime rather than perl-native

[Dexuan Cui]

Signed-off-by: Dexuan Cui <dexuan.cui@intel.com>
---
 meta/recipes-sato/webkit/webkit-gtk_svn.bb |    6 +++---
 1 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/meta/recipes-sato/webkit/webkit-gtk_svn.bb b/meta/recipes-sato/webkit/webkit-gtk_svn.bb
index 3c5101b..ac0e683 100644
--- a/meta/recipes-sato/webkit/webkit-gtk_svn.bb
+++ b/meta/recipes-sato/webkit/webkit-gtk_svn.bb
@@ -7,14 +7,14 @@ LIC_FILES_CHKSUM = "file://WebCore/rendering/RenderApplet.h;endline=22;md5=fb969
                     file://WebKit/gtk/webkit/webkit.h;endline=21;md5=b4fbe9f4a944f1d071dba1d2c76b3351 \
                     file://JavaScriptCore/parser/Parser.h;endline=23;md5=2f3cff0ad0a9c486da5a376928973a90"
 
-DEPENDS = "enchant gnome-keyring libsoup-2.4 curl icu libxml2 cairo libxslt libxt libidn gnutls gtk+ gstreamer gst-plugins-base gnome-vfs flex-native gperf-native perl-native sqlite3"
-DEPENDS_darwin8 = "curl icu libxml2 cairo libxslt libidn gnutls gtk+ gstreamer flex-native gperf-native perl-native sqlite3"
+DEPENDS = "enchant gnome-keyring libsoup-2.4 curl icu libxml2 cairo libxslt libxt libidn gnutls gtk+ gstreamer gst-plugins-base gnome-vfs flex-native gperf-native perl-native-runtime sqlite3"
+DEPENDS_darwin8 = "curl icu libxml2 cairo libxslt libidn gnutls gtk+ gstreamer flex-native gperf-native perl-native-runtime sqlite3"
 
 SRCREV_FORMAT = "webcore-rwebkit"
 
 SRCREV = "72836"
 PV = "1.3.7+svnr${SRCPV}"
-PR = "r0"
+PR = "r1"
 
 SRC_URI = "\
   svn://svn.webkit.org/repository/webkit/trunk/;module=JavaScriptCore;proto=http \
-- 
1.7.2

[RFC v1 PATCH 11/16] perl: inherit perlnative

[Dexuan Cui]

Signed-off-by: Dexuan Cui <dexuan.cui@intel.com>
---
 meta/recipes-devtools/perl/perl_5.12.3.bb |   10 +++++-----
 1 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/meta/recipes-devtools/perl/perl_5.12.3.bb b/meta/recipes-devtools/perl/perl_5.12.3.bb
index 847ee59..b053482 100644
--- a/meta/recipes-devtools/perl/perl_5.12.3.bb
+++ b/meta/recipes-devtools/perl/perl_5.12.3.bb
@@ -6,9 +6,9 @@ LIC_FILES_CHKSUM = "file://Copying;md5=2b4c6ffbcfcbdee469f02565f253d81a \
 		    file://Artistic;md5=f921793d03cc6d63ec4b15e9be8fd3f8"
 PRIORITY = "optional"
 # We need gnugrep (for -I)
-DEPENDS = "virtual/db perl-native-${PV} grep-native"
+DEPENDS = "virtual/db grep-native"
 DEPENDS += "gdbm zlib"
-PR = "r0"
+PR = "r1"
 
 # 5.10.1 has Module::Build built-in
 PROVIDES += "libmodule-build-perl"
@@ -81,13 +81,13 @@ SRC_URI = "http://www.cpan.org/src/5.0/perl-${PV}.tar.gz \
 SRC_URI[md5sum] = "29975a69dce54e47fcd6331c085c6c99"
 SRC_URI[sha256sum] = "5678bfd5c2cd59253a26171bf3e681235433b00c730eea8a8046e1b225c11d2f"
 
-inherit siteinfo
+inherit perlnative siteinfo
 
 # Where to find the native perl
-HOSTPERL = "${STAGING_BINDIR_NATIVE}/perl${PV}"
+HOSTPERL = "${STAGING_BINDIR_NATIVE}/perl-native/perl${PV}"
 
 # Where to find .so files - use the -native versions not those from the target build
-export PERLHOSTLIB = "${STAGING_LIBDIR_NATIVE}/perl/${PV}/"
+export PERLHOSTLIB = "${STAGING_LIBDIR_NATIVE}/perl-native/perl/${PV}/"
 
 # LDFLAGS for shared libraries
 export LDDLFLAGS = "${LDFLAGS} -shared"
-- 
1.7.2

[RFC v1 PATCH 12/16] cpan.bbclass, cpan-base.bbclas: update them for the perlnative change

[Dexuan Cui]

Since perl-native now populates into its own dir, here we need
change accordingly.

Signed-off-by: Dexuan Cui <dexuan.cui@intel.com>
---
 meta/classes/cpan-base.bbclass |   10 +++++++---
 meta/classes/cpan.bbclass      |   12 ++++++------
 2 files changed, 13 insertions(+), 9 deletions(-)

diff --git a/meta/classes/cpan-base.bbclass b/meta/classes/cpan-base.bbclass
index 6cd2aa3..b5dbdae 100644
--- a/meta/classes/cpan-base.bbclass
+++ b/meta/classes/cpan-base.bbclass
@@ -7,10 +7,12 @@ FILES_${PN} += "${libdir}/perl ${datadir}/perl"
 DEPENDS  += "${@["perl", "perl-native"][(bb.data.inherits_class('native', d))]}"
 RDEPENDS  += "${@["perl", ""][(bb.data.inherits_class('native', d))]}"
 
+PERL_OWN_DIR = "${@["", "/perl-native"][(bb.data.inherits_class('native', d))]}"
+
 # Determine the staged version of perl from the perl configuration file
 def get_perl_version(d):
 	import re
-    	cfg = bb.data.expand('${STAGING_LIBDIR}/perl/config.sh', d)
+    	cfg = bb.data.expand('${STAGING_LIBDIR}${PERL_OWN_DIR}/perl/config.sh', d)
 	try:
 		f = open(cfg, 'r')
 	except IOError:
@@ -27,8 +29,10 @@ def get_perl_version(d):
 # Determine where the library directories are
 def perl_get_libdirs(d):
 	libdir = bb.data.getVar('libdir', d, 1)
-	libdirs = libdir + '/perl'
-	return libdirs
+	if is_target(d) == "no":
+		libdir += '/perl-native'
+	libdir += '/perl'
+	return libdir
 
 def is_target(d):
     if not bb.data.inherits_class('native', d):
diff --git a/meta/classes/cpan.bbclass b/meta/classes/cpan.bbclass
index 9b8431b..cbf428d 100644
--- a/meta/classes/cpan.bbclass
+++ b/meta/classes/cpan.bbclass
@@ -1,7 +1,7 @@
 #
 # This is for perl modules that use the old Makefile.PL build system
 #
-inherit cpan-base
+inherit cpan-base perlnative
 
 EXTRA_CPANFLAGS ?= ""
 EXTRA_PERLFLAGS ?= ""
@@ -10,16 +10,16 @@ EXTRA_PERLFLAGS ?= ""
 export PERLCONFIGTARGET = "${@is_target(d)}"
 
 # Env var which tells perl where the perl include files are
-export PERL_INC = "${STAGING_LIBDIR}/perl/${@get_perl_version(d)}/CORE"
-export PERL_LIB = "${STAGING_LIBDIR}/perl/${@get_perl_version(d)}"
-export PERL_ARCHLIB = "${STAGING_LIBDIR}/perl/${@get_perl_version(d)}"
-export PERLHOSTLIB = "${STAGING_LIBDIR_NATIVE}/perl/${@get_perl_version(d)}/"
+export PERL_INC = "${STAGING_LIBDIR}${PERL_OWN_DIR}/perl/${@get_perl_version(d)}/CORE"
+export PERL_LIB = "${STAGING_LIBDIR}${PERL_OWN_DIR}/perl/${@get_perl_version(d)}"
+export PERL_ARCHLIB = "${STAGING_LIBDIR}${PERL_OWN_DIR}/perl/${@get_perl_version(d)}"
+export PERLHOSTLIB = "${STAGING_LIBDIR_NATIVE}/perl-native/perl/${@get_perl_version(d)}/"
 
 cpan_do_configure () {
 	export PERL5LIB="${PERL_ARCHLIB}"
 	yes '' | perl ${EXTRA_PERLFLAGS} Makefile.PL ${EXTRA_CPANFLAGS}
 	if [ "${BUILD_SYS}" != "${HOST_SYS}" ]; then
-		. ${STAGING_LIBDIR}/perl/config.sh
+		. ${STAGING_LIBDIR}${PERL_OWN_DIR}/perl/config.sh
 		# Use find since there can be a Makefile generated for each Makefile.PL
 		for f in `find -name Makefile.PL`; do
 			f2=`echo $f | sed -e 's/.PL//'`
-- 
1.7.2

[RFC v1 PATCH 13/16] libxml-parser-perl: inherit perlnative

[Dexuan Cui]

Signed-off-by: Dexuan Cui <dexuan.cui@intel.com>
---
 .../perl/libxml-parser-perl-native_2.40.bb         |    2 +-
 .../perl/libxml-parser-perl_2.40.bb                |    2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/meta/recipes-devtools/perl/libxml-parser-perl-native_2.40.bb b/meta/recipes-devtools/perl/libxml-parser-perl-native_2.40.bb
index 6ba56b3..8c20455 100644
--- a/meta/recipes-devtools/perl/libxml-parser-perl-native_2.40.bb
+++ b/meta/recipes-devtools/perl/libxml-parser-perl-native_2.40.bb
@@ -4,4 +4,4 @@ require libxml-parser-perl_${PV}.bb
 
 inherit native
 
-DEPENDS = "expat-native perl-native"
\ No newline at end of file
+DEPENDS += "expat-native"
diff --git a/meta/recipes-devtools/perl/libxml-parser-perl_2.40.bb b/meta/recipes-devtools/perl/libxml-parser-perl_2.40.bb
index c2d1bdb..fac28a8 100644
--- a/meta/recipes-devtools/perl/libxml-parser-perl_2.40.bb
+++ b/meta/recipes-devtools/perl/libxml-parser-perl_2.40.bb
@@ -3,7 +3,7 @@ LICENSE = "Artistic"
 LIC_FILES_CHKSUM = "file://README;beginline=2;endline=6;md5=c8767d7516229f07b26e42d1cf8b51f1"
 DEPENDS += "expat expat-native"
 
-PR = "r0"
+PR = "r1"
 
 SRC_URI = "http://www.cpan.org/modules/by-module/XML/XML-Parser-${PV}.tar.gz"
 
-- 
1.7.2

[RFC v1 PATCH 14/16] libconvert-asn1-perl: fix EXTRA_PERLFLAGS due to the perl-native change

[Dexuan Cui]

Signed-off-by: Dexuan Cui <dexuan.cui@intel.com>
---
 .../perl/libconvert-asn1-perl_0.22.bb              |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/meta/recipes-extended/perl/libconvert-asn1-perl_0.22.bb b/meta/recipes-extended/perl/libconvert-asn1-perl_0.22.bb
index e3edc18..fc77b75 100644
--- a/meta/recipes-extended/perl/libconvert-asn1-perl_0.22.bb
+++ b/meta/recipes-extended/perl/libconvert-asn1-perl_0.22.bb
@@ -2,7 +2,7 @@ DESCRIPTION = "Convert::ASN1 - ASN.1 Encode/Decode library"
 SECTION = "libs"
 LICENSE = "Artistic|GPLv1+"
 LIC_FILES_CHKSUM = "file://README;beginline=10;endline=12;md5=a64b291b13ffddeef33b14f047ee8b26"
-PR = "r0"
+PR = "r1"
 
 SRC_URI = "http://search.cpan.org/CPAN/authors/id/G/GB/GBARR/Convert-ASN1-${PV}.tar.gz"
 
@@ -13,7 +13,7 @@ S = "${WORKDIR}/Convert-ASN1-${PV}"
 
 inherit cpan
 
-EXTRA_PERLFLAGS = "-I ${STAGING_LIBDIR_NATIVE}/perl/${@get_perl_version(d)}"
+EXTRA_PERLFLAGS = "-I ${STAGING_LIBDIR_NATIVE}/perl-native/perl/${@get_perl_version(d)}"
 
 BBCLASSEXTEND="native"
 
-- 
1.7.2

[RFC v1 PATCH 15/16] libxml-simple-perl: fix EXTRA_PERLFLAGS due the the perlnative change

[Dexuan Cui]

Signed-off-by: Dexuan Cui <dexuan.cui@intel.com>
---
 .../perl/libxml-simple-perl-native_2.18.bb         |    2 +-
 .../perl/libxml-simple-perl_2.18.bb                |    4 ++--
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/meta/recipes-devtools/perl/libxml-simple-perl-native_2.18.bb b/meta/recipes-devtools/perl/libxml-simple-perl-native_2.18.bb
index bfdfc3d..3877d0b 100644
--- a/meta/recipes-devtools/perl/libxml-simple-perl-native_2.18.bb
+++ b/meta/recipes-devtools/perl/libxml-simple-perl-native_2.18.bb
@@ -4,4 +4,4 @@ inherit native
 
 require libxml-simple-perl_${PV}.bb
 
-DEPENDS = "libxml-parser-perl-native perl-native"
+DEPENDS += "libxml-parser-perl-native"
diff --git a/meta/recipes-devtools/perl/libxml-simple-perl_2.18.bb b/meta/recipes-devtools/perl/libxml-simple-perl_2.18.bb
index ca57776..89b65b2 100644
--- a/meta/recipes-devtools/perl/libxml-simple-perl_2.18.bb
+++ b/meta/recipes-devtools/perl/libxml-simple-perl_2.18.bb
@@ -2,7 +2,7 @@ SECTION = "libs"
 LICENSE = "Artistic"
 LIC_FILES_CHKSUM = "file://README;beginline=70;md5=94aa5d46682b411a53a5494cfb22640e"
 DEPENDS += "libxml-parser-perl"
-PR = "r1"
+PR = "r2"
 
 SRC_URI = "http://www.cpan.org/modules/by-module/XML/XML-Simple-${PV}.tar.gz"
 
@@ -11,6 +11,6 @@ SRC_URI[sha256sum] = "a54967c188cda3e20f496c83be4de3f1740eeaa83c0380712ecd969ad8
 
 S = "${WORKDIR}/XML-Simple-${PV}"
 
-EXTRA_PERLFLAGS = "-I ${STAGING_LIBDIR_NATIVE}/perl/${@get_perl_version(d)}"
+EXTRA_PERLFLAGS = "-I ${STAGING_LIBDIR_NATIVE}/perl-native/perl/${@get_perl_version(d)}"
 
 inherit cpan
-- 
1.7.2

[RFC v1 PATCH 16/16] icon-naming-utils-native: inherit perlnative

[Dexuan Cui]

Signed-off-by: Dexuan Cui <dexuan.cui@intel.com>
---
 .../icon-naming-utils-native_0.8.7.bb              |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/meta/recipes-devtools/icon-naming-utils/icon-naming-utils-native_0.8.7.bb b/meta/recipes-devtools/icon-naming-utils/icon-naming-utils-native_0.8.7.bb
index 18aa374..d34fb03 100644
--- a/meta/recipes-devtools/icon-naming-utils/icon-naming-utils-native_0.8.7.bb
+++ b/meta/recipes-devtools/icon-naming-utils/icon-naming-utils-native_0.8.7.bb
@@ -1,6 +1,6 @@
 LICENSE = "GPLv2"
 DEPENDS = "libxml-simple-perl-native"
-PR = "r1"
+PR = "r2"
 
 LIC_FILES_CHKSUM = "file://COPYING;md5=94d55d512a9ba36caa9b7df079bae19f"
 
@@ -8,4 +8,4 @@ SRC_URI = "http://tango.freedesktop.org/releases/icon-naming-utils-0.8.7.tar.gz"
 
 S = "${WORKDIR}/icon-naming-utils-${PV}"
 
-inherit autotools native
+inherit autotools native perlnative
-- 
1.7.2

Re: [RFC v1 PATCH 00/16] populate perl-native into its own directory

[Richard Purdie]

Hi Dexuan,

On Wed, 2011-06-01 at 21:18 +0800, Dexuan Cui wrote:
> Currently both perl-native (a.k.a. ${STAGING_BINDIR_NATIVE}/perl )and
> perl-native-runtime(a.k.a. the system perl, or /usr/bin/perl) appear in
> the PATH when bitbake is running.
> This can cause some race conditions: many places detecting perl from PATH
> can't make sure which path will be used as this depends on when perl-native's
> populate_sysroot is finished, e.g., automake-native and autoconf-native could
> use perl-native-runtime while gnu-config-native could use perl-native and
> this inconsistent usages can cause trouble, e.g., bug 941.
> 
> And, as RP suggested, "the time to use perl-native instead of
> perl-native-runtime is where any perl module is being built, perl itself is
> being built or anything that has an explicit dependency on the perl version
> present".
> 
> So I made the following changes to try to address the aboves issues:
> 1) populate perl-native into its own directory so it won't appear in PATH
> by default, and add perlnative.bbclass for these recipes that really depend
> on perl-native;
> 2) check all perl-native references and correct the ones that should be
> perl-native-runtime;
> 3) fix any building issues due to the new location of perl-native,
> especially cpan and cpan-base .bbclass.
> 
> With the changes, bug 941 doesn't appear.
> 
> Tests I did are:
> I tried "bitbale core-image-sato-sdk and meta-toolchain-gmae" in x86_32 and
> x86_64 Ubuntu hosts and everything seems building fine.
> 
> 
> Please review the changes and comment on them. Thanks!

I had a look through the series and it looks good to me. Hopefully this
addresses the perl issues people have been seeing once and for all (and
unclogs the dependencies a little) :)

I'm going to leave this on the mailing list for a while to give others a
chance to review it though.

Cheers,

Richard

Re: [RFC v1 PATCH 00/16] populate perl-native into its own directory

[Tom Rini]

On 06/01/2011 06:18 AM, Dexuan Cui wrote:
> Currently both perl-native (a.k.a. ${STAGING_BINDIR_NATIVE}/perl )and
> perl-native-runtime(a.k.a. the system perl, or /usr/bin/perl) appear in
> the PATH when bitbake is running.
> This can cause some race conditions: many places detecting perl from PATH
> can't make sure which path will be used as this depends on when perl-native's
> populate_sysroot is finished, e.g., automake-native and autoconf-native could
> use perl-native-runtime while gnu-config-native could use perl-native and
> this inconsistent usages can cause trouble, e.g., bug 941.
> 
> And, as RP suggested, "the time to use perl-native instead of
> perl-native-runtime is where any perl module is being built, perl itself is
> being built or anything that has an explicit dependency on the perl version
> present".
> 
> So I made the following changes to try to address the aboves issues:
> 1) populate perl-native into its own directory so it won't appear in PATH
> by default, and add perlnative.bbclass for these recipes that really depend
> on perl-native;
> 2) check all perl-native references and correct the ones that should be
> perl-native-runtime;
> 3) fix any building issues due to the new location of perl-native,
> especially cpan and cpan-base .bbclass.
> 
> With the changes, bug 941 doesn't appear.
> 
> Tests I did are:
> I tried "bitbale core-image-sato-sdk and meta-toolchain-gmae" in x86_32 and
> x86_64 Ubuntu hosts and everything seems building fine.

How does this work (by which I mean, test some please :)) with meta-oe
where we have (or will have soon) a lot of perl module recipes?  My
concern is that we've just moved the race around a bit and we'll hit
this somewhere else where we both really need "perl-native" (since we
need some cpan stuff we've built) and also mangle in the perl we found
in PATH into some scripts...

-- 
Tom Rini
Mentor Graphics Corporation

Re: [RFC v1 PATCH 00/16] populate perl-native into its own directory

[Richard Purdie]

On Wed, 2011-06-01 at 10:17 -0700, Tom Rini wrote:
> On 06/01/2011 06:18 AM, Dexuan Cui wrote:
> > Currently both perl-native (a.k.a. ${STAGING_BINDIR_NATIVE}/perl )and
> > perl-native-runtime(a.k.a. the system perl, or /usr/bin/perl) appear in
> > the PATH when bitbake is running.
> > This can cause some race conditions: many places detecting perl from PATH
> > can't make sure which path will be used as this depends on when perl-native's
> > populate_sysroot is finished, e.g., automake-native and autoconf-native could
> > use perl-native-runtime while gnu-config-native could use perl-native and
> > this inconsistent usages can cause trouble, e.g., bug 941.
> > 
> > And, as RP suggested, "the time to use perl-native instead of
> > perl-native-runtime is where any perl module is being built, perl itself is
> > being built or anything that has an explicit dependency on the perl version
> > present".
> > 
> > So I made the following changes to try to address the aboves issues:
> > 1) populate perl-native into its own directory so it won't appear in PATH
> > by default, and add perlnative.bbclass for these recipes that really depend
> > on perl-native;
> > 2) check all perl-native references and correct the ones that should be
> > perl-native-runtime;
> > 3) fix any building issues due to the new location of perl-native,
> > especially cpan and cpan-base .bbclass.
> > 
> > With the changes, bug 941 doesn't appear.
> > 
> > Tests I did are:
> > I tried "bitbale core-image-sato-sdk and meta-toolchain-gmae" in x86_32 and
> > x86_64 Ubuntu hosts and everything seems building fine.
> 
> How does this work (by which I mean, test some please :)) with meta-oe
> where we have (or will have soon) a lot of perl module recipes?  My
> concern is that we've just moved the race around a bit and we'll hit
> this somewhere else where we both really need "perl-native" (since we
> need some cpan stuff we've built) and also mangle in the perl we found
> in PATH into some scripts...

Anything building or using perl modules should be inheriting the
perlnative class. This adds the dependency on perl-native and the
appropriate PATH entry. Where is the race?

Cheers,

Richard

Re: [RFC v1 PATCH 00/16] populate perl-native into its own directory

[Tom Rini]

On 06/01/2011 10:56 AM, Richard Purdie wrote:
> On Wed, 2011-06-01 at 10:17 -0700, Tom Rini wrote:
>> On 06/01/2011 06:18 AM, Dexuan Cui wrote:
>>> Currently both perl-native (a.k.a. ${STAGING_BINDIR_NATIVE}/perl )and
>>> perl-native-runtime(a.k.a. the system perl, or /usr/bin/perl) appear in
>>> the PATH when bitbake is running.
>>> This can cause some race conditions: many places detecting perl from PATH
>>> can't make sure which path will be used as this depends on when perl-native's
>>> populate_sysroot is finished, e.g., automake-native and autoconf-native could
>>> use perl-native-runtime while gnu-config-native could use perl-native and
>>> this inconsistent usages can cause trouble, e.g., bug 941.
>>>
>>> And, as RP suggested, "the time to use perl-native instead of
>>> perl-native-runtime is where any perl module is being built, perl itself is
>>> being built or anything that has an explicit dependency on the perl version
>>> present".
>>>
>>> So I made the following changes to try to address the aboves issues:
>>> 1) populate perl-native into its own directory so it won't appear in PATH
>>> by default, and add perlnative.bbclass for these recipes that really depend
>>> on perl-native;
>>> 2) check all perl-native references and correct the ones that should be
>>> perl-native-runtime;
>>> 3) fix any building issues due to the new location of perl-native,
>>> especially cpan and cpan-base .bbclass.
>>>
>>> With the changes, bug 941 doesn't appear.
>>>
>>> Tests I did are:
>>> I tried "bitbale core-image-sato-sdk and meta-toolchain-gmae" in x86_32 and
>>> x86_64 Ubuntu hosts and everything seems building fine.
>>
>> How does this work (by which I mean, test some please :)) with meta-oe
>> where we have (or will have soon) a lot of perl module recipes?  My
>> concern is that we've just moved the race around a bit and we'll hit
>> this somewhere else where we both really need "perl-native" (since we
>> need some cpan stuff we've built) and also mangle in the perl we found
>> in PATH into some scripts...
> 
> Anything building or using perl modules should be inheriting the
> perlnative class. This adds the dependency on perl-native and the
> appropriate PATH entry. Where is the race?

Maybe race isn't quite the right word.  But recipe A depends on
lib$something-perl-native, and brings in perl-native.  It also checks
for perl in its auto-foo and finds our perl.  It now also uses our perl
when it wants a host perl and all of the potential bad things happen, yes?

-- 
Tom Rini
Mentor Graphics Corporation

Re: [RFC v1 PATCH 00/16] populate perl-native into its own directory

[Phil Blundell]

On Wed, 2011-06-01 at 12:39 -0700, Tom Rini wrote:
> Maybe race isn't quite the right word.  But recipe A depends on
> lib$something-perl-native, and brings in perl-native.  It also checks
> for perl in its auto-foo and finds our perl.  It now also uses our perl
> when it wants a host perl and all of the potential bad things happen, yes?

What are the circumstances in which it would want a host perl?  I don't
quite understand the issue here.  Surely anything that the host perl can
do, perl-native plus some combination of libfoo-perl-native can also do,
right?

p.

Re: [RFC v1 PATCH 00/16] populate perl-native into its own directory

[Tom Rini]

On 06/01/2011 01:05 PM, Phil Blundell wrote:
> On Wed, 2011-06-01 at 12:39 -0700, Tom Rini wrote:
>> Maybe race isn't quite the right word.  But recipe A depends on
>> lib$something-perl-native, and brings in perl-native.  It also checks
>> for perl in its auto-foo and finds our perl.  It now also uses our perl
>> when it wants a host perl and all of the potential bad things happen, yes?
> 
> What are the circumstances in which it would want a host perl?  I don't
> quite understand the issue here.  Surely anything that the host perl can
> do, perl-native plus some combination of libfoo-perl-native can also do,
> right?

So, here's the two things going on:
- In some cases (libfoo-perl-native) we need to install various perl
modules in order to build other target apps.  This is the cpan case,
iirc.  We don't currently (and can't?) just play whatever games perl
would like to do to use system-wide perl and a local to TMPDIR cpan
directory.  We instead go down the path of having perl-native.
- In order to build perl for the target we need to have the same perl
version available for use.

What we do in oe-core (and oe.dev, historically) is provide perl-native
for both of these cases.  What falls down in this case is that  once
perl-native is built (and in our PATH), if it's a different version than
system-wide perl, stuff starts failing on version mis-match.  The patch
series here fixes that by saying stuff must opt-in to having perl-native
be in PATH rather than just system-wide perl.  What I'm saying is that
while in oe-core nothing falls down here (since nothing that gets
perl-native also tries to just run 'perl') meta-oe is or will have failures.

Unfortunately I don't have the test cases that drove me to make oe.dev
like it is handy but I have a feeling that if we go down this path it
won't be the last we see of the problem, but it might well pop up again.

There is another option here, which is to make perl-native be
perl-cross, live in its own special directory and figure out how to make
a TMPDIR-specific cpan repository available for use by system-wide perl.

-- 
Tom Rini
Mentor Graphics Corporation

Re: [RFC v1 PATCH 00/16] populate perl-native into its own directory

[Phil Blundell]

On Wed, 2011-06-01 at 13:42 -0700, Tom Rini wrote:
> What falls down in this case is that  once
> perl-native is built (and in our PATH), if it's a different version than
> system-wide perl, stuff starts failing on version mis-match.

I think that's the bit that I'm not properly understanding.  Which
versions are mismatching, exactly?  

Surely the local perl from the sysroot ought to be completely
self-contained and shouldn't be using any bits from the host perl
install at that point.

p.

Re: [RFC v1 PATCH 00/16] populate perl-native into its own directory

[Tom Rini]

On 06/01/2011 01:45 PM, Phil Blundell wrote:
> On Wed, 2011-06-01 at 13:42 -0700, Tom Rini wrote:
>> What falls down in this case is that  once
>> perl-native is built (and in our PATH), if it's a different version than
>> system-wide perl, stuff starts failing on version mis-match.
> 
> I think that's the bit that I'm not properly understanding.  Which
> versions are mismatching, exactly?  
> 
> Surely the local perl from the sysroot ought to be completely
> self-contained and shouldn't be using any bits from the host perl
> install at that point.

So this jogs my memory a bit!  It's not so much perl itself but stuff
that uses perl that can get dirty and then no, you have stuff thats
built for system perl and stuff that's built with perl-native clashing.

Relying even more on memory, I think help2man was one of the "easy"
culprits and since we also modify the env, we do things like have
help2man run with PERL5LIB and so on pointing system-wide perl at
perl-native's lib directory and so forth.

-- 
Tom Rini
Mentor Graphics Corporation

Re: [RFC v1 PATCH 00/16] populate perl-native into its own directory

[Richard Purdie]

On Wed, 2011-06-01 at 13:59 -0700, Tom Rini wrote:
> On 06/01/2011 01:45 PM, Phil Blundell wrote:
> > On Wed, 2011-06-01 at 13:42 -0700, Tom Rini wrote:
> >> What falls down in this case is that  once
> >> perl-native is built (and in our PATH), if it's a different version than
> >> system-wide perl, stuff starts failing on version mis-match.
> > 
> > I think that's the bit that I'm not properly understanding.  Which
> > versions are mismatching, exactly?  
> > 
> > Surely the local perl from the sysroot ought to be completely
> > self-contained and shouldn't be using any bits from the host perl
> > install at that point.
> 
> So this jogs my memory a bit!  It's not so much perl itself but stuff
> that uses perl that can get dirty and then no, you have stuff thats
> built for system perl and stuff that's built with perl-native clashing.
> 
> Relying even more on memory, I think help2man was one of the "easy"
> culprits and since we also modify the env, we do things like have
> help2man run with PERL5LIB and so on pointing system-wide perl at
> perl-native's lib directory and so forth.

But with the proposed patch series either:

a) help2man depends perlnative.bbclass

In this case it can depend on perl-native being there, its in path and
things work as per OE.dev.

b) help2man doesn't depend in perlnative.bbclass

It only sees the system perl.

So I'm still not clear where the problem is?

Cheers,

Richard

Re: [RFC v1 PATCH 00/16] populate perl-native into its own directory

[Tom Rini]

On 06/02/2011 07:06 AM, Richard Purdie wrote:
> On Wed, 2011-06-01 at 13:59 -0700, Tom Rini wrote:
>> On 06/01/2011 01:45 PM, Phil Blundell wrote:
>>> On Wed, 2011-06-01 at 13:42 -0700, Tom Rini wrote:
>>>> What falls down in this case is that  once
>>>> perl-native is built (and in our PATH), if it's a different version than
>>>> system-wide perl, stuff starts failing on version mis-match.
>>>
>>> I think that's the bit that I'm not properly understanding.  Which
>>> versions are mismatching, exactly?  
>>>
>>> Surely the local perl from the sysroot ought to be completely
>>> self-contained and shouldn't be using any bits from the host perl
>>> install at that point.
>>
>> So this jogs my memory a bit!  It's not so much perl itself but stuff
>> that uses perl that can get dirty and then no, you have stuff thats
>> built for system perl and stuff that's built with perl-native clashing.
>>
>> Relying even more on memory, I think help2man was one of the "easy"
>> culprits and since we also modify the env, we do things like have
>> help2man run with PERL5LIB and so on pointing system-wide perl at
>> perl-native's lib directory and so forth.
> 
> But with the proposed patch series either:
> 
> a) help2man depends perlnative.bbclass
> 
> In this case it can depend on perl-native being there, its in path and
> things work as per OE.dev.
> 
> b) help2man doesn't depend in perlnative.bbclass
> 
> It only sees the system perl.
> 
> So I'm still not clear where the problem is?

Well, help2man-native is (or needs to be) a dep listed in
autotools.bbclass (since so many things need it, hence why it oe-core
still just has it as a required host utility instead of building it).

But help2man is just the easy/common case.  Heck, it _may_ blow up even
with the host help2man instead of help2man-native, if a recipe uses
system-wide help2man and perlnative.bbclass.  The root problem (again,
from memory) is that since we modify PERL5LIB and so on, when we do
that, we've opened ourselves up for system-wide perl trying to use
perl-native's stuff.

-- 
Tom Rini
Mentor Graphics Corporation

Re: [RFC v1 PATCH 00/16] populate perl-native into its own directory

[Phil Blundell]

On Thu, 2011-06-02 at 07:25 -0700, Tom Rini wrote:
> But help2man is just the easy/common case.  Heck, it _may_ blow up even
> with the host help2man instead of help2man-native, if a recipe uses
> system-wide help2man and perlnative.bbclass.  The root problem (again,
> from memory) is that since we modify PERL5LIB and so on, when we do
> that, we've opened ourselves up for system-wide perl trying to use
> perl-native's stuff.

Ah right, I think I see what you're getting at now.

If we've got a clean separation between perl-native and host perl,
though, can't we now just eliminate all of that futzing with PERL5LIB in
cpan.bbclass and such like places?  perl-native already knows how to
look in the right places within the sysroot for its modules so there
should be no need for anything else to be overriding it.

p.

Re: [RFC v1 PATCH 00/16] populate perl-native into its own directory

[Tom Rini]

On 06/02/2011 07:37 AM, Phil Blundell wrote:
> On Thu, 2011-06-02 at 07:25 -0700, Tom Rini wrote:
>> But help2man is just the easy/common case.  Heck, it _may_ blow up even
>> with the host help2man instead of help2man-native, if a recipe uses
>> system-wide help2man and perlnative.bbclass.  The root problem (again,
>> from memory) is that since we modify PERL5LIB and so on, when we do
>> that, we've opened ourselves up for system-wide perl trying to use
>> perl-native's stuff.
> 
> Ah right, I think I see what you're getting at now.
> 
> If we've got a clean separation between perl-native and host perl,
> though, can't we now just eliminate all of that futzing with PERL5LIB in
> cpan.bbclass and such like places?  perl-native already knows how to
> look in the right places within the sysroot for its modules so there
> should be no need for anything else to be overriding it.

Well, my question is, does it, really?  Even if we're using the sstate
cache from /foo/oecore/tmp over in /bar/oecore/tmp (and /foo/oecore/tmp
is rm -rf'd) ?  Since we've got a create_wrapper around perl and
perl${PV} it should be I suppose (or is easily added there), but I'd
feel a lot better with some testing of the above case (and the updates
to cpan*bbclass).

-- 
Tom Rini
Mentor Graphics Corporation

Re: [RFC v1 PATCH 00/16] populate perl-native into its own directory

[Richard Purdie]

On Thu, 2011-06-02 at 09:28 -0700, Tom Rini wrote:
> On 06/02/2011 07:37 AM, Phil Blundell wrote:
> > On Thu, 2011-06-02 at 07:25 -0700, Tom Rini wrote:
> >> But help2man is just the easy/common case.  Heck, it _may_ blow up even
> >> with the host help2man instead of help2man-native, if a recipe uses
> >> system-wide help2man and perlnative.bbclass.  The root problem (again,
> >> from memory) is that since we modify PERL5LIB and so on, when we do
> >> that, we've opened ourselves up for system-wide perl trying to use
> >> perl-native's stuff.
> > 
> > Ah right, I think I see what you're getting at now.
> > 
> > If we've got a clean separation between perl-native and host perl,
> > though, can't we now just eliminate all of that futzing with PERL5LIB in
> > cpan.bbclass and such like places?  perl-native already knows how to
> > look in the right places within the sysroot for its modules so there
> > should be no need for anything else to be overriding it.
> 
> Well, my question is, does it, really?

If it doesn't it really does need to get fixed. I've not seen any
evidence this has been the problem but it still could be.

>   Even if we're using the sstate
> cache from /foo/oecore/tmp over in /bar/oecore/tmp (and /foo/oecore/tmp
> is rm -rf'd) ?  Since we've got a create_wrapper around perl and
> perl${PV} it should be I suppose (or is easily added there), but I'd
> feel a lot better with some testing of the above case (and the updates
> to cpan*bbclass).

I only took the perl-native DEPENDS patch on the condition this gets
fixed properly. The patches that are there look to do that, at least for
OE-Core. If there are further issues we're going to have to take them as
they arise as I have an objection to crippling the build dependencies
because perl is broken. Really this could use some TLC from someone with
experience in the area...

Cheers,

Richard

Re: [RFC v1 PATCH 00/16] populate perl-native into its own directory

[Tom Rini]

On 06/02/2011 09:35 AM, Richard Purdie wrote:
> On Thu, 2011-06-02 at 09:28 -0700, Tom Rini wrote:
>> On 06/02/2011 07:37 AM, Phil Blundell wrote:
>>> On Thu, 2011-06-02 at 07:25 -0700, Tom Rini wrote:
>>>> But help2man is just the easy/common case.  Heck, it _may_ blow up even
>>>> with the host help2man instead of help2man-native, if a recipe uses
>>>> system-wide help2man and perlnative.bbclass.  The root problem (again,
>>>> from memory) is that since we modify PERL5LIB and so on, when we do
>>>> that, we've opened ourselves up for system-wide perl trying to use
>>>> perl-native's stuff.
>>>
>>> Ah right, I think I see what you're getting at now.
>>>
>>> If we've got a clean separation between perl-native and host perl,
>>> though, can't we now just eliminate all of that futzing with PERL5LIB in
>>> cpan.bbclass and such like places?  perl-native already knows how to
>>> look in the right places within the sysroot for its modules so there
>>> should be no need for anything else to be overriding it.
>>
>> Well, my question is, does it, really?
> 
> If it doesn't it really does need to get fixed. I've not seen any
> evidence this has been the problem but it still could be.

I agree it needs to be fixed, yes.  One way or another...

>>   Even if we're using the sstate
>> cache from /foo/oecore/tmp over in /bar/oecore/tmp (and /foo/oecore/tmp
>> is rm -rf'd) ?  Since we've got a create_wrapper around perl and
>> perl${PV} it should be I suppose (or is easily added there), but I'd
>> feel a lot better with some testing of the above case (and the updates
>> to cpan*bbclass).
> 
> I only took the perl-native DEPENDS patch on the condition this gets
> fixed properly. The patches that are there look to do that, at least for
> OE-Core. If there are further issues we're going to have to take them as
> they arise as I have an objection to crippling the build dependencies
> because perl is broken. Really this could use some TLC from someone with
> experience in the area...

Well, I guess I'd boil down what I said above into a request like this
for v3:
- Modify cpan.bbclass to NOT set PERL_INC / PERL_LIB / PERL_ARCHLIB /
PERLHOSTLIB.
- In /scratch/oecore/tmp0 build the images that were built for v1
- In /scratch/oecore/tmp1 build perl-native's full sstate cache.
- Keep the sstate cache from tmp1 and otherwise rm -rf tmp1.
- In /scratch/oecore/tmp2 using the sstate from tmp1, build the same
images that were built for tmp0.

If all of that works, I think we can call this safe enough for now and
possibly even really fixed, while we have someone actively kicking
around the recipes in question.

-- 
Tom Rini
Mentor Graphics Corporation

Re: [RFC v1 PATCH 00/16] populate perl-native into its own directory

[Richard Purdie]

On Thu, 2011-06-02 at 09:55 -0700, Tom Rini wrote:
> On 06/02/2011 09:35 AM, Richard Purdie wrote:
> > On Thu, 2011-06-02 at 09:28 -0700, Tom Rini wrote:
> >>   Even if we're using the sstate
> >> cache from /foo/oecore/tmp over in /bar/oecore/tmp (and /foo/oecore/tmp
> >> is rm -rf'd) ?  Since we've got a create_wrapper around perl and
> >> perl${PV} it should be I suppose (or is easily added there), but I'd
> >> feel a lot better with some testing of the above case (and the updates
> >> to cpan*bbclass).
> > 
> > I only took the perl-native DEPENDS patch on the condition this gets
> > fixed properly. The patches that are there look to do that, at least for
> > OE-Core. If there are further issues we're going to have to take them as
> > they arise as I have an objection to crippling the build dependencies
> > because perl is broken. Really this could use some TLC from someone with
> > experience in the area...
> 
> Well, I guess I'd boil down what I said above into a request like this
> for v3:
> - Modify cpan.bbclass to NOT set PERL_INC / PERL_LIB / PERL_ARCHLIB /
> PERLHOSTLIB.

The first three of these are all about the *target* perl location and I
think we still need them due the mess that perl's build system is. With
the patch series in question they won't actually point at perl-native in
the target case and they are only really used for cross compiling
purposes.

PERLHOSTLIB is used by the target perl when cross compiling to find
native .so files. perl-native will always be present at this point and
again, it seems like a valid use case.

Summary is that I don't think perl-native is broken in any way but we do
need those variables.

> - In /scratch/oecore/tmp0 build the images that were built for v1
> - In /scratch/oecore/tmp1 build perl-native's full sstate cache.
> - Keep the sstate cache from tmp1 and otherwise rm -rf tmp1.
> - In /scratch/oecore/tmp2 using the sstate from tmp1, build the same
> images that were built for tmp0.

I'm confused by this test cycle. What do you mean by "build
perl-native's full sstate cache"?

I suspect you've asking for some partial sstate cache to be shared
between two builds?

Put simpler, you probably want:

in tmp0 "bitbake perl-native"
in tmp1, different location to tmp0, "bitbake core-image-sato" but
sharing the same sstate cache

Is that what you're thinking?

FWIW, I've been running locally with the patch series and I think I'd
like to merge it. If there are issues we can address them as and when
they're identified...

Cheers,

Richard

Re: [RFC v1 PATCH 00/16] populate perl-native into its own directory

[Richard Purdie]

On Thu, 2011-06-09 at 09:04 +0100, Richard Purdie wrote:
> On Thu, 2011-06-02 at 09:55 -0700, Tom Rini wrote:
> > On 06/02/2011 09:35 AM, Richard Purdie wrote:
> > > On Thu, 2011-06-02 at 09:28 -0700, Tom Rini wrote:
> > >>   Even if we're using the sstate
> > >> cache from /foo/oecore/tmp over in /bar/oecore/tmp (and /foo/oecore/tmp
> > >> is rm -rf'd) ?  Since we've got a create_wrapper around perl and
> > >> perl${PV} it should be I suppose (or is easily added there), but I'd
> > >> feel a lot better with some testing of the above case (and the updates
> > >> to cpan*bbclass).
> > > 
> > > I only took the perl-native DEPENDS patch on the condition this gets
> > > fixed properly. The patches that are there look to do that, at least for
> > > OE-Core. If there are further issues we're going to have to take them as
> > > they arise as I have an objection to crippling the build dependencies
> > > because perl is broken. Really this could use some TLC from someone with
> > > experience in the area...
> > 
> > Well, I guess I'd boil down what I said above into a request like this
> > for v3:
> > - Modify cpan.bbclass to NOT set PERL_INC / PERL_LIB / PERL_ARCHLIB /
> > PERLHOSTLIB.
> 
> The first three of these are all about the *target* perl location and I
> think we still need them due the mess that perl's build system is. With
> the patch series in question they won't actually point at perl-native in
> the target case and they are only really used for cross compiling
> purposes.
> 
> PERLHOSTLIB is used by the target perl when cross compiling to find
> native .so files. perl-native will always be present at this point and
> again, it seems like a valid use case.
> 
> Summary is that I don't think perl-native is broken in any way but we do
> need those variables.
> 
> > - In /scratch/oecore/tmp0 build the images that were built for v1
> > - In /scratch/oecore/tmp1 build perl-native's full sstate cache.
> > - Keep the sstate cache from tmp1 and otherwise rm -rf tmp1.
> > - In /scratch/oecore/tmp2 using the sstate from tmp1, build the same
> > images that were built for tmp0.
> 
> I'm confused by this test cycle. What do you mean by "build
> perl-native's full sstate cache"?
> 
> I suspect you've asking for some partial sstate cache to be shared
> between two builds?
> 
> Put simpler, you probably want:
> 
> in tmp0 "bitbake perl-native"
> in tmp1, different location to tmp0, "bitbake core-image-sato" but
> sharing the same sstate cache

I meant to add that tmp0 should be renamed before this second step so if
there are hardcoded references in any of the sstate packages they
couldn't find anything in tmp0 as it would no longer exist.

> Is that what you're thinking?
> 
> FWIW, I've been running locally with the patch series and I think I'd
> like to merge it. If there are issues we can address them as and when
> they're identified...
> 
> Cheers,
> 
> Richard
> 
> 
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core

Re: [RFC v1 PATCH 00/16] populate perl-native into its own directory

[Cui, Dexuan]

Richard Purdie wrote:
> On Thu, 2011-06-09 at 09:04 +0100, Richard Purdie wrote:
>> On Thu, 2011-06-02 at 09:55 -0700, Tom Rini wrote:
>>> On 06/02/2011 09:35 AM, Richard Purdie wrote:
>>>> On Thu, 2011-06-02 at 09:28 -0700, Tom Rini wrote:
>>>>>   Even if we're using the sstate
>>>>> cache from /foo/oecore/tmp over in /bar/oecore/tmp (and
>>>>> /foo/oecore/tmp is rm -rf'd) ?  Since we've got a create_wrapper
>>>>> around perl and perl${PV} it should be I suppose (or is easily
>>>>> added there), but I'd feel a lot better with some testing of the
>>>>> above case (and the updates to cpan*bbclass).
>>>> 
>>>> I only took the perl-native DEPENDS patch on the condition this
>>>> gets fixed properly. The patches that are there look to do that,
>>>> at least for OE-Core. If there are further issues we're going to
>>>> have to take them as they arise as I have an objection to
>>>> crippling the build dependencies because perl is broken. Really
>>>> this could use some TLC from someone with experience in the area...
>>> 
>>> Well, I guess I'd boil down what I said above into a request like
>>> this for v3: - Modify cpan.bbclass to NOT set PERL_INC / PERL_LIB /
>>> PERL_ARCHLIB / PERLHOSTLIB.
>> 
>> The first three of these are all about the *target* perl location
>> and I think we still need them due the mess that perl's build system
>> is. With the patch series in question they won't actually point at
>> perl-native in the target case and they are only really used for
>> cross compiling purposes. 
>> 
>> PERLHOSTLIB is used by the target perl when cross compiling to find
>> native .so files. perl-native will always be present at this point
>> and again, it seems like a valid use case.
>> 
>> Summary is that I don't think perl-native is broken in any way but
>> we do need those variables. 
>> 
>>> - In /scratch/oecore/tmp0 build the images that were built for v1
>>> - In /scratch/oecore/tmp1 build perl-native's full sstate cache.
>>> - Keep the sstate cache from tmp1 and otherwise rm -rf tmp1.
>>> - In /scratch/oecore/tmp2 using the sstate from tmp1, build the same
>>> images that were built for tmp0.
>> 
>> I'm confused by this test cycle. What do you mean by "build
>> perl-native's full sstate cache"?
>> 
>> I suspect you've asking for some partial sstate cache to be shared
>> between two builds? 
>> 
>> Put simpler, you probably want:
>> 
>> in tmp0 "bitbake perl-native"
>> in tmp1, different location to tmp0, "bitbake core-image-sato" but
>> sharing the same sstate cache
> 
> I meant to add that tmp0 should be renamed before this second step so
> if there are hardcoded references in any of the sstate packages they
> couldn't find anything in tmp0 as it would no longer exist.
Actually this doesn't work even without my patch series?
i.e., without my patch series,
1) I "bitbake perl-native" in /poky.git/build/;
2) mv /poky.git/build /poky.git/build.bak;
3) in /poky.git/build2/, modify the conf/local.conf to set SSTATE_MIRRORS to point to /poky.git/build.bak/sstate-cache/, and "bitbake sgmlspl-native" would fail in do_compile:
"Can't locate ExtUtils/Command.pm in @INC (@INC contains: /poky.git/build/tmp/sysroots/i686-linux/usr/lib/perl/5.12.3 /poky.git/build/tmp/sysroots/i686-linux/usr/lib/perl/5.12.3 /poky.git/build/tmp/sysroots/i686-"linux/usr/lib/perl/5.12.3"
Is this an existing bug? 

Thanks,
-- Dexuan

Re: [RFC v1 PATCH 00/16] populate perl-native into its own directory

[Richard Purdie]

On Thu, 2011-06-09 at 21:51 +0800, Cui, Dexuan wrote:
> Richard Purdie wrote:
> > On Thu, 2011-06-09 at 09:04 +0100, Richard Purdie wrote: 
> >> I suspect you've asking for some partial sstate cache to be shared
> >> between two builds? 
> >> 
> >> Put simpler, you probably want:
> >> 
> >> in tmp0 "bitbake perl-native"
> >> in tmp1, different location to tmp0, "bitbake core-image-sato" but
> >> sharing the same sstate cache
> > 
> > I meant to add that tmp0 should be renamed before this second step so
> > if there are hardcoded references in any of the sstate packages they
> > couldn't find anything in tmp0 as it would no longer exist.
>
> Actually this doesn't work even without my patch series?
> i.e., without my patch series,
> 1) I "bitbake perl-native" in /poky.git/build/;
> 2) mv /poky.git/build /poky.git/build.bak;
> 3) in /poky.git/build2/, modify the conf/local.conf to set SSTATE_MIRRORS to point to /poky.git/build.bak/sstate-cache/, and "bitbake sgmlspl-native" would fail in do_compile:
> "Can't locate ExtUtils/Command.pm in @INC (@INC contains: /poky.git/build/tmp/sysroots/i686-linux/usr/lib/perl/5.12.3 /poky.git/build/tmp/sysroots/i686-linux/usr/lib/perl/5.12.3 /poky.git/build/tmp/sysroots/i686-"linux/usr/lib/perl/5.12.3"
> Is this an existing bug? 

Looks like it. This did work but it looks like its been broken
somehow :(

Cheers,

Richard

Re: [RFC v1 PATCH 00/16] populate perl-native into its own directory

[Cui, Dexuan]

Richard Purdie wrote:
> On Thu, 2011-06-09 at 21:51 +0800, Cui, Dexuan wrote:
>> Richard Purdie wrote:
>>> On Thu, 2011-06-09 at 09:04 +0100, Richard Purdie wrote:
>>>> I suspect you've asking for some partial sstate cache to be shared
>>>> between two builds? 
>>>> 
>>>> Put simpler, you probably want:
>>>> 
>>>> in tmp0 "bitbake perl-native"
>>>> in tmp1, different location to tmp0, "bitbake core-image-sato" but
>>>> sharing the same sstate cache
>>> 
>>> I meant to add that tmp0 should be renamed before this second step
>>> so if there are hardcoded references in any of the sstate packages
>>> they couldn't find anything in tmp0 as it would no longer exist.
>> 
>> Actually this doesn't work even without my patch series?
>> i.e., without my patch series,
>> 1) I "bitbake perl-native" in /poky.git/build/;
>> 2) mv /poky.git/build /poky.git/build.bak;
>> 3) in /poky.git/build2/, modify the conf/local.conf to set
>> SSTATE_MIRRORS to point to /poky.git/build.bak/sstate-cache/, and
>> "bitbake sgmlspl-native" would fail in do_compile: "Can't locate
>> ExtUtils/Command.pm in @INC (@INC contains:
>> /poky.git/build/tmp/sysroots/i686-linux/usr/lib/perl/5.12.3
>> /poky.git/build/tmp/sysroots/i686-linux/usr/lib/perl/5.12.3
>> /poky.git/build/tmp/sysroots/i686-"linux/usr/lib/perl/5.12.3" Is
>> this an existing bug?     
> 
> Looks like it. This did work but it looks like its been broken
> somehow :(
> 
Hi Richard,
I reported http://bugzilla.yoctoproject.org/show_bug.cgi?id=1157 to track this issue.

Thanks,
-- Dexuan

Re: [RFC v1 PATCH 00/16] populate perl-native into its own directory

[Tom Rini]

On 06/09/2011 01:08 AM, Richard Purdie wrote:
> On Thu, 2011-06-09 at 09:04 +0100, Richard Purdie wrote:
>> On Thu, 2011-06-02 at 09:55 -0700, Tom Rini wrote:
>>> On 06/02/2011 09:35 AM, Richard Purdie wrote:
>>>> On Thu, 2011-06-02 at 09:28 -0700, Tom Rini wrote:
>>>>>   Even if we're using the sstate
>>>>> cache from /foo/oecore/tmp over in /bar/oecore/tmp (and /foo/oecore/tmp
>>>>> is rm -rf'd) ?  Since we've got a create_wrapper around perl and
>>>>> perl${PV} it should be I suppose (or is easily added there), but I'd
>>>>> feel a lot better with some testing of the above case (and the updates
>>>>> to cpan*bbclass).
>>>>
>>>> I only took the perl-native DEPENDS patch on the condition this gets
>>>> fixed properly. The patches that are there look to do that, at least for
>>>> OE-Core. If there are further issues we're going to have to take them as
>>>> they arise as I have an objection to crippling the build dependencies
>>>> because perl is broken. Really this could use some TLC from someone with
>>>> experience in the area...
>>>
>>> Well, I guess I'd boil down what I said above into a request like this
>>> for v3:
>>> - Modify cpan.bbclass to NOT set PERL_INC / PERL_LIB / PERL_ARCHLIB /
>>> PERLHOSTLIB.
>>
>> The first three of these are all about the *target* perl location and I
>> think we still need them due the mess that perl's build system is. With
>> the patch series in question they won't actually point at perl-native in
>> the target case and they are only really used for cross compiling
>> purposes.
>>
>> PERLHOSTLIB is used by the target perl when cross compiling to find
>> native .so files. perl-native will always be present at this point and
>> again, it seems like a valid use case.
>>
>> Summary is that I don't think perl-native is broken in any way but we do
>> need those variables.
>>
>>> - In /scratch/oecore/tmp0 build the images that were built for v1
>>> - In /scratch/oecore/tmp1 build perl-native's full sstate cache.
>>> - Keep the sstate cache from tmp1 and otherwise rm -rf tmp1.
>>> - In /scratch/oecore/tmp2 using the sstate from tmp1, build the same
>>> images that were built for tmp0.
>>
>> I'm confused by this test cycle. What do you mean by "build
>> perl-native's full sstate cache"?
>>
>> I suspect you've asking for some partial sstate cache to be shared
>> between two builds?
>>
>> Put simpler, you probably want:
>>
>> in tmp0 "bitbake perl-native"
>> in tmp1, different location to tmp0, "bitbake core-image-sato" but
>> sharing the same sstate cache
> 
> I meant to add that tmp0 should be renamed before this second step so if
> there are hardcoded references in any of the sstate packages they
> couldn't find anything in tmp0 as it would no longer exist.

Yes, this is what I'm asking for.

-- 
Tom Rini
Mentor Graphics Corporation

Re: [RFC v1 PATCH 00/16] populate perl-native into its own directory

[Tom Rini]

On 06/09/2011 01:04 AM, Richard Purdie wrote:
> On Thu, 2011-06-02 at 09:55 -0700, Tom Rini wrote:
>> On 06/02/2011 09:35 AM, Richard Purdie wrote:
>>> On Thu, 2011-06-02 at 09:28 -0700, Tom Rini wrote:
>>>>   Even if we're using the sstate
>>>> cache from /foo/oecore/tmp over in /bar/oecore/tmp (and /foo/oecore/tmp
>>>> is rm -rf'd) ?  Since we've got a create_wrapper around perl and
>>>> perl${PV} it should be I suppose (or is easily added there), but I'd
>>>> feel a lot better with some testing of the above case (and the updates
>>>> to cpan*bbclass).
>>>
>>> I only took the perl-native DEPENDS patch on the condition this gets
>>> fixed properly. The patches that are there look to do that, at least for
>>> OE-Core. If there are further issues we're going to have to take them as
>>> they arise as I have an objection to crippling the build dependencies
>>> because perl is broken. Really this could use some TLC from someone with
>>> experience in the area...
>>
>> Well, I guess I'd boil down what I said above into a request like this
>> for v3:
>> - Modify cpan.bbclass to NOT set PERL_INC / PERL_LIB / PERL_ARCHLIB /
>> PERLHOSTLIB.
> 
> The first three of these are all about the *target* perl location and I
> think we still need them due the mess that perl's build system is. With
> the patch series in question they won't actually point at perl-native in
> the target case and they are only really used for cross compiling
> purposes.
> 
> PERLHOSTLIB is used by the target perl when cross compiling to find
> native .so files. perl-native will always be present at this point and
> again, it seems like a valid use case.
> 
> Summary is that I don't think perl-native is broken in any way but we do
> need those variables.

Maybe I'm having a dense morning here, but isn't that the point?  The
combination of perl's build system is messy and if we bring in cpan it
needs not only target perl locations (to dump things into) but may call
'perl' to do things and if we're on perl 5.14 and your host is only 5.10
or 5.12, bad things can happen.  And since cpan.bbclass isn't brought in
by target perl recipe, what now? :)

-- 
Tom Rini
Mentor Graphics Corporation

Re: [RFC v1 PATCH 00/16] populate perl-native into its own directory

[Cui, Dexuan]

Tom Rini wrote:
> On 06/01/2011 01:05 PM, Phil Blundell wrote:
>> On Wed, 2011-06-01 at 12:39 -0700, Tom Rini wrote:
>>> Maybe race isn't quite the right word.  But recipe A depends on
>>> lib$something-perl-native, and brings in perl-native.  It also
>>> checks 
>>> for perl in its auto-foo and finds our perl.  It now also uses our
>>> perl when it wants a host perl and all of the potential bad things
>>> happen, yes? 
>> 
>> What are the circumstances in which it would want a host perl?  I
>> don't quite understand the issue here.  Surely anything that the
>> host perl can do, perl-native plus some combination of
>> libfoo-perl-native can also do, right?
> 
> So, here's the two things going on:
> - In some cases (libfoo-perl-native) we need to install various perl
> modules in order to build other target apps.  This is the cpan case,
> iirc.  We don't currently (and can't?) just play whatever games perl
> would like to do to use system-wide perl and a local to TMPDIR cpan
> directory.  We instead go down the path of having perl-native.
> - In order to build perl for the target we need to have the same perl
> version available for use.
> 
> What we do in oe-core (and oe.dev, historically) is provide
> perl-native 
> for both of these cases.  What falls down in this case is that  once
> perl-native is built (and in our PATH), if it's a different version
> than system-wide perl, stuff starts failing on version mis-match. 
> The patch series here fixes that by saying stuff must opt-in to
> having perl-native 
> be in PATH rather than just system-wide perl.  What I'm saying is that
> while in oe-core nothing falls down here (since nothing that gets
> perl-native also tries to just run 'perl') meta-oe is or will have
> failures. 
> 
> Unfortunately I don't have the test cases that drove me to make oe.dev
> like it is handy but I have a feeling that if we go down this path it
> won't be the last we see of the problem, but it might well pop up
> again. 
> 
> There is another option here, which is to make perl-native be
> perl-cross, live in its own special directory and figure out how to
> make 
> a TMPDIR-specific cpan repository available for use by system-wide
> perl. 

Hi Tom,
Thank you very much for the detailed explanation!
However I'm actually new to cpan and know few about the history of that in oe.dev, so I'm not sure I know enough about the background to make a proper decision now.
I suppose Richard would comment here. :-)

Thanks,
-- Dexuan