[PATCH 0/2] add nativesdk to BBCLASSEXTEND to python-distribute and to fix the side effect of it.

[PATCH 0/2] add nativesdk to BBCLASSEXTEND to python-distribute and to fix the side effect of it.

[João Henrique Ferreira de Freitas]

Hi, 

My company has a private python build tool that has python-distribute as dependency.
That tool builds well with -native. And now I need to put the my private tool in toolchain. So
my users could use it.

To accomplish, I added nativesdk to BBCLASSEXTEND. My buildtools-mytool recipe builds and the
toolchain get created. But when I try to install the toolchain I get the this:

$ tmp/deploy/sdk/buildtools-mytool-x86_64-nativesdk-standalone-1.6.1.0.sh 
Enter target directory for SDK (default: /opt/mydistro/mytool/1.6.1.0): 
You are about to install the SDK to "/opt/mydistro/mytool/1.6.1.0". Proceed[Y/n]?
Extracting SDK...done
Setting it up.../opt/mydistro/mytool/1.6.1.0/relocate_sdk.sh: linha 2: erro de sintaxe próximo do `token' não esperado `('

One of the files that relocate_sdk.sh was trying to handle is:

/opt/mydistro/mytool/1.6.1.0/sysroots/x
86_64-linuxsdk-linux/usr/lib/python2.7/site-packages/distribute-0.6.32-py2.7.egg/setuptools/script template (dev).py

Maybe there are other recipes when adding nativesdk to BBCLASSEXTEND which this special handler is needed. So it's better 
ensure the correct handler of file name with spaces characters.

João Henrique Ferreira de Freitas (2):
  python-distribute: add nativesdk to BBCLASSEXTEND
  populate_sdk_base: ensure that filenames with empty space character
    are handled

 meta/classes/populate_sdk_base.bbclass                   | 4 ++--
 meta/recipes-devtools/python/python-distribute_0.6.32.bb | 7 ++++++-
 2 files changed, 8 insertions(+), 3 deletions(-)

-- 
1.9.1

[PATCH 1/2] python-distribute: add nativesdk to BBCLASSEXTEND

[João Henrique Ferreira de Freitas]

Without this, python tools that need python-distribute doesn't install
in SDK generated by OE.

Signed-off-by: João Henrique Ferreira de Freitas <joaohf@gmail.com>
---
 meta/recipes-devtools/python/python-distribute_0.6.32.bb | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/meta/recipes-devtools/python/python-distribute_0.6.32.bb b/meta/recipes-devtools/python/python-distribute_0.6.32.bb
index 58bb8b4..d576a4c 100644
--- a/meta/recipes-devtools/python/python-distribute_0.6.32.bb
+++ b/meta/recipes-devtools/python/python-distribute_0.6.32.bb
@@ -39,6 +39,11 @@ do_install_append() {
     echo "./${SRCNAME}-${PV}-py${PYTHON_BASEVERSION}.egg" > ${D}${PYTHON_SITEPACKAGES_DIR}/setuptools.pth
 }
 
+do_install_append_class-nativesdk() {
+	sed -i -e '1s|^#!.*|#!/usr/bin/env python|' ${D}${PYTHON_SITEPACKAGES_DIR}/distribute-${PV}-py${PYTHON_BASEVERSION}.egg/setuptools/tests/test_resources.py
+	sed -i -e '1s|^#!.*|#!/usr/bin/env python|' ${D}${bindir}/easy_install
+}
+
 RDEPENDS_${PN} = "\
   python-distutils \
   python-compression \
@@ -46,4 +51,4 @@ RDEPENDS_${PN} = "\
 
 RPROVIDES_${PN} += "python-setuptools"
 
-BBCLASSEXTEND = "native"
+BBCLASSEXTEND = "native nativesdk"
-- 
1.9.1

[PATCH 2/2] populate_sdk_base: ensure that filenames with empty space character are handled

[João Henrique Ferreira de Freitas]

When extracting toolchain, if the list $executable_files has filenames
with empty space character, the list will created but relocate_sdk.sh
will not handle it well. This will lead to the below erro:

    ./tmp/deploy/sdk/buildtools-mytools-x86_64-nativesdk-standalone-1.6.1.0.sh
    Enter target directory for SDK (default: /opt/mydistro/mytoolset/1.6.1.0):
    You are about to install the SDK to "/opt/mydistro/mytoolset/1.6.1.0". Proceed[Y/n]?
    Extracting SDK...done
    Setting it up.../opt/mydistro/mytoolset/1.6.1.0/relocate_sdk.sh: line 2: sintaxe error `token'  `('
    /opt/mydistro/mytoolset/1.6.1.0/relocate_sdk.sh: line 2: `e

The same occurs with replacement of ${SDKPATH} in configs/scripts/etc files.

We should ensure that full path is protected before relocate_sdk.sh
and ${SDKPATH} replacement calls.

Signed-off-by: João Henrique Ferreira de Freitas <joaohf@gmail.com>
---
 meta/classes/populate_sdk_base.bbclass | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/meta/classes/populate_sdk_base.bbclass b/meta/classes/populate_sdk_base.bbclass
index a12bf11..5311271 100644
--- a/meta/classes/populate_sdk_base.bbclass
+++ b/meta/classes/populate_sdk_base.bbclass
@@ -252,7 +252,7 @@ if [ "$dl_path" = "" ] ; then
 	echo "SDK could not be set up. Relocate script unable to find ld-linux.so. Abort!"
 	exit 1
 fi
-executable_files=$($SUDO_EXEC find $native_sysroot -type f -perm /111)
+executable_files=$($SUDO_EXEC find $native_sysroot -type f -perm /111 -exec echo \'{}\' \;)
 
 tdir=`mktemp -d`
 if [ x$tdir = x ] ; then
@@ -273,7 +273,7 @@ if [ $relocate = 1 ] ; then
 fi
 
 # replace ${SDKPATH} with the new prefix in all text files: configs/scripts/etc
-$SUDO_EXEC find $native_sysroot -type f -exec file '{}' \;|grep ":.*\(ASCII\|script\|source\).*text"|cut -d':' -f1|$SUDO_EXEC xargs sed -i -e "s:$DEFAULT_INSTALL_DIR:$target_sdk_dir:g"
+$SUDO_EXEC find $native_sysroot -type f -exec file '{}' \;|grep ":.*\(ASCII\|script\|source\).*text"|cut -d':' -f1|tr "\n" "\0"|$SUDO_EXEC xargs -0 sed -i -e "s:$DEFAULT_INSTALL_DIR:$target_sdk_dir:g"
 
 # change all symlinks pointing to ${SDKPATH}
 for l in $($SUDO_EXEC find $native_sysroot -type l); do
-- 
1.9.1

Re: [PATCH 2/2] populate_sdk_base: ensure that filenames with empty space character are handled

[Saul Wold]

On 07/01/2014 06:58 PM, João Henrique Ferreira de Freitas wrote:
> When extracting toolchain, if the list $executable_files has filenames
> with empty space character, the list will created but relocate_sdk.sh
> will not handle it well. This will lead to the below erro:
>
>      ./tmp/deploy/sdk/buildtools-mytools-x86_64-nativesdk-standalone-1.6.1.0.sh
>      Enter target directory for SDK (default: /opt/mydistro/mytoolset/1.6.1.0):
>      You are about to install the SDK to "/opt/mydistro/mytoolset/1.6.1.0". Proceed[Y/n]?
>      Extracting SDK...done
>      Setting it up.../opt/mydistro/mytoolset/1.6.1.0/relocate_sdk.sh: line 2: sintaxe error `token'  `('
>      /opt/mydistro/mytoolset/1.6.1.0/relocate_sdk.sh: line 2: `e
>
> The same occurs with replacement of ${SDKPATH} in configs/scripts/etc files.
>
> We should ensure that full path is protected before relocate_sdk.sh
> and ${SDKPATH} replacement calls.
>

I am not sure about this one, there are other cases where spaces in 
filenames/directory pathes just don't work, so fixing one place may not 
address all issues, I thought this was documented someplace, but I can't 
seem to find the associated bug or wiki entries.

Maybe someone else can remind me also where we talk about whitespace in 
pathe, obviously my search-fu is off right now.

Sau!

> Signed-off-by: João Henrique Ferreira de Freitas <joaohf@gmail.com>
> ---
>   meta/classes/populate_sdk_base.bbclass | 4 ++--
>   1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/meta/classes/populate_sdk_base.bbclass b/meta/classes/populate_sdk_base.bbclass
> index a12bf11..5311271 100644
> --- a/meta/classes/populate_sdk_base.bbclass
> +++ b/meta/classes/populate_sdk_base.bbclass
> @@ -252,7 +252,7 @@ if [ "$dl_path" = "" ] ; then
>   	echo "SDK could not be set up. Relocate script unable to find ld-linux.so. Abort!"
>   	exit 1
>   fi
> -executable_files=$($SUDO_EXEC find $native_sysroot -type f -perm /111)
> +executable_files=$($SUDO_EXEC find $native_sysroot -type f -perm /111 -exec echo \'{}\' \;)
>
>   tdir=`mktemp -d`
>   if [ x$tdir = x ] ; then
> @@ -273,7 +273,7 @@ if [ $relocate = 1 ] ; then
>   fi
>
>   # replace ${SDKPATH} with the new prefix in all text files: configs/scripts/etc
> -$SUDO_EXEC find $native_sysroot -type f -exec file '{}' \;|grep ":.*\(ASCII\|script\|source\).*text"|cut -d':' -f1|$SUDO_EXEC xargs sed -i -e "s:$DEFAULT_INSTALL_DIR:$target_sdk_dir:g"
> +$SUDO_EXEC find $native_sysroot -type f -exec file '{}' \;|grep ":.*\(ASCII\|script\|source\).*text"|cut -d':' -f1|tr "\n" "\0"|$SUDO_EXEC xargs -0 sed -i -e "s:$DEFAULT_INSTALL_DIR:$target_sdk_dir:g"
>
>   # change all symlinks pointing to ${SDKPATH}
>   for l in $($SUDO_EXEC find $native_sysroot -type l); do
>

Re: [PATCH 2/2] populate_sdk_base: ensure that filenames with empty space character are handled

[Otavio Salvador]

Hello Saul,

On Tue, Jul 8, 2014 at 9:19 PM, Saul Wold <sgw@linux.intel.com> wrote:
> On 07/01/2014 06:58 PM, João Henrique Ferreira de Freitas wrote:
>>
>> When extracting toolchain, if the list $executable_files has filenames
>> with empty space character, the list will created but relocate_sdk.sh
>> will not handle it well. This will lead to the below erro:
>>
>>
>> ./tmp/deploy/sdk/buildtools-mytools-x86_64-nativesdk-standalone-1.6.1.0.sh
>>      Enter target directory for SDK (default:
>> /opt/mydistro/mytoolset/1.6.1.0):
>>      You are about to install the SDK to
>> "/opt/mydistro/mytoolset/1.6.1.0". Proceed[Y/n]?
>>      Extracting SDK...done
>>      Setting it up.../opt/mydistro/mytoolset/1.6.1.0/relocate_sdk.sh: line
>> 2: sintaxe error `token'  `('
>>      /opt/mydistro/mytoolset/1.6.1.0/relocate_sdk.sh: line 2: `e
>>
>> The same occurs with replacement of ${SDKPATH} in configs/scripts/etc
>> files.
>>
>> We should ensure that full path is protected before relocate_sdk.sh
>> and ${SDKPATH} replacement calls.
>>
>
> I am not sure about this one, there are other cases where spaces in
> filenames/directory pathes just don't work, so fixing one place may not
> address all issues, I thought this was documented someplace, but I can't
> seem to find the associated bug or wiki entries.
>
> Maybe someone else can remind me also where we talk about whitespace in
> pathe, obviously my search-fu is off right now.

I agree this problem should exist in other places but I think this
does not invalidate Joao's patch and it should be reviewed and merged
if correct.

Now talking about  the patch itself, Joao, did you try to avoid the
extra fork calling echo? Using '-print0' option in find might help.

-- 
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://code.ossystems.com.br
Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750

Re: [PATCH 2/2] populate_sdk_base: ensure that filenames with empty space character are handled

[João Henrique Ferreira de Freitas]

Hi Otavio

Em 08-07-2014 22:33, Otavio Salvador escreveu:
> Now talking about the patch itself, Joao, did you try to avoid the 
> extra fork calling echo? Using '-print0' option in find might help. 

Not until now. But I agree about your concern. So, instead of

executable_files=$($SUDO_EXEC find $native_sysroot -type f -perm /111 
-exec echo \'{}\' \;)

Its better do:

executable_files=$($SUDO_EXEC find $native_sysroot -type f -perm /111 
-printf "'%h/%f' ")

No extra fork was used with -printf "'%h/%f' ".

The argument -print0 is not an option because the output of find command 
will be 'path1 path2 path 3' a concatenation of strings not strings 
split by space and protected with ' '.

Both results are valid. But the second is nicer. What do you think?


When I did this patch I was not thinking to solve all problems about 
spaces in filename. However a deep dig is needed to track all points 
that have this issue.

Thanks.

-- 
João Henrique Ferreira de Freitas - joaohf_at_gmail.com
Campinas-SP-Brasil

Re: [PATCH 2/2] populate_sdk_base: ensure that filenames with empty space character are handled

[Otavio Salvador]

On Sat, Jul 12, 2014 at 10:20 AM, João Henrique Ferreira de Freitas
<joaohf@gmail.com> wrote:
>
> Hi Otavio
>
> Em 08-07-2014 22:33, Otavio Salvador escreveu:
>
>> Now talking about the patch itself, Joao, did you try to avoid the extra
>> fork calling echo? Using '-print0' option in find might help.
>
>
> Not until now. But I agree about your concern. So, instead of
>
>
> executable_files=$($SUDO_EXEC find $native_sysroot -type f -perm /111 -exec
> echo \'{}\' \;)
>
> Its better do:
>
> executable_files=$($SUDO_EXEC find $native_sysroot -type f -perm /111
> -printf "'%h/%f' ")
>
> No extra fork was used with -printf "'%h/%f' ".
>
> The argument -print0 is not an option because the output of find command
> will be 'path1 path2 path 3' a concatenation of strings not strings split by
> space and protected with ' '.
>
> Both results are valid. But the second is nicer. What do you think?

Nice.

> When I did this patch I was not thinking to solve all problems about spaces
> in filename. However a deep dig is needed to track all points that have this
> issue.

Fully agree but this shouldn't deny this patch to be applied. It is a
step further.

-- 
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://code.ossystems.com.br
Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750

[PATCH v2 0/2] add nativesdk to BBCLASSEXTEND to python-distribute and to fix the side effect of it.

[João Henrique Ferreira de Freitas]

Changes in v2:

 - Use 'find -print' instead of an extra fork to protect path
 
 - Replace sha-bang for easy_python version variant

Test steps:

1- apply the two patches

2- include the package 'nativesdk-python-distribute to TOOLCHAIN_HOST_TASK in meta/recipes-core/meta/buildtools-tarball.bb

3- run 'bitbake buildtools-tarball'

4- install the created SDK

Files from python-distribute has space in three filenames and the SDK step will not claim about it.

João Henrique Ferreira de Freitas (2):
  python-distribute: add nativesdk to BBCLASSEXTEND
  populate_sdk_base: ensure that filenames with empty space character
    are handled

 meta/classes/populate_sdk_base.bbclass                   | 4 ++--
 meta/recipes-devtools/python/python-distribute_0.6.32.bb | 8 +++++++-
 2 files changed, 9 insertions(+), 3 deletions(-)

-- 
1.9.1

[PATCH v2 1/2] python-distribute: add nativesdk to BBCLASSEXTEND

[João Henrique Ferreira de Freitas]

Without this, python tools that need python-distribute doesn't install
in SDK generated by OE.

Signed-off-by: João Henrique Ferreira de Freitas <joaohf@gmail.com>
---
 meta/recipes-devtools/python/python-distribute_0.6.32.bb | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/meta/recipes-devtools/python/python-distribute_0.6.32.bb b/meta/recipes-devtools/python/python-distribute_0.6.32.bb
index 58bb8b4..7000145 100644
--- a/meta/recipes-devtools/python/python-distribute_0.6.32.bb
+++ b/meta/recipes-devtools/python/python-distribute_0.6.32.bb
@@ -39,6 +39,12 @@ do_install_append() {
     echo "./${SRCNAME}-${PV}-py${PYTHON_BASEVERSION}.egg" > ${D}${PYTHON_SITEPACKAGES_DIR}/setuptools.pth
 }
 
+do_install_append_class-nativesdk() {
+	sed -i -e '1s|^#!.*|#!/usr/bin/env python|' ${D}${PYTHON_SITEPACKAGES_DIR}/distribute-${PV}-py${PYTHON_BASEVERSION}.egg/setuptools/tests/test_resources.py
+	sed -i -e '1s|^#!.*|#!/usr/bin/env python|' ${D}${bindir}/easy_install
+	sed -i -e '1s|^#!.*|#!/usr/bin/env python|' ${D}${bindir}/easy_install-${PYTHON_BASEVERSION}
+}
+
 RDEPENDS_${PN} = "\
   python-distutils \
   python-compression \
@@ -46,4 +52,4 @@ RDEPENDS_${PN} = "\
 
 RPROVIDES_${PN} += "python-setuptools"
 
-BBCLASSEXTEND = "native"
+BBCLASSEXTEND = "native nativesdk"
-- 
1.9.1

[PATCH v2 2/2] populate_sdk_base: ensure that filenames with empty space character are handled

[João Henrique Ferreira de Freitas]

When extracting toolchain, if the list $executable_files has filenames
with empty space character, the list will created but relocate_sdk.sh
will not handle it well. This will lead to the below erro:

    ./tmp/deploy/sdk/buildtools-mytools-x86_64-nativesdk-standalone-1.6.1.0.sh
    Enter target directory for SDK (default: /opt/mydistro/mytoolset/1.6.1.0):
    You are about to install the SDK to "/opt/mydistro/mytoolset/1.6.1.0". Proceed[Y/n]?
    Extracting SDK...done
    Setting it up.../opt/mydistro/mytoolset/1.6.1.0/relocate_sdk.sh: line 2: sintaxe error `token'  `('
    /opt/mydistro/mytoolset/1.6.1.0/relocate_sdk.sh: line 2: `e

The same occurs with replacement of ${SDKPATH} in configs/scripts/etc files.

We should ensure that full path is protected before relocate_sdk.sh
and ${SDKPATH} replacement calls.

Signed-off-by: João Henrique Ferreira de Freitas <joaohf@gmail.com>
---
 meta/classes/populate_sdk_base.bbclass | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/meta/classes/populate_sdk_base.bbclass b/meta/classes/populate_sdk_base.bbclass
index a12bf11..a6e23a1 100644
--- a/meta/classes/populate_sdk_base.bbclass
+++ b/meta/classes/populate_sdk_base.bbclass
@@ -252,7 +252,7 @@ if [ "$dl_path" = "" ] ; then
 	echo "SDK could not be set up. Relocate script unable to find ld-linux.so. Abort!"
 	exit 1
 fi
-executable_files=$($SUDO_EXEC find $native_sysroot -type f -perm /111)
+executable_files=$($SUDO_EXEC find $native_sysroot -type f -perm /111 -printf "'%h/%f' ")
 
 tdir=`mktemp -d`
 if [ x$tdir = x ] ; then
@@ -273,7 +273,7 @@ if [ $relocate = 1 ] ; then
 fi
 
 # replace ${SDKPATH} with the new prefix in all text files: configs/scripts/etc
-$SUDO_EXEC find $native_sysroot -type f -exec file '{}' \;|grep ":.*\(ASCII\|script\|source\).*text"|cut -d':' -f1|$SUDO_EXEC xargs sed -i -e "s:$DEFAULT_INSTALL_DIR:$target_sdk_dir:g"
+$SUDO_EXEC find $native_sysroot -type f -exec file '{}' \;|grep ":.*\(ASCII\|script\|source\).*text"|cut -d':' -f1|tr "\n" "\0"|$SUDO_EXEC xargs -0 sed -i -e "s:$DEFAULT_INSTALL_DIR:$target_sdk_dir:g"
 
 # change all symlinks pointing to ${SDKPATH}
 for l in $($SUDO_EXEC find $native_sysroot -type l); do
-- 
1.9.1

Re: [PATCH v2 2/2] populate_sdk_base: ensure that filenames with empty space character are handled

[Otavio Salvador]

On Mon, Jul 14, 2014 at 7:55 PM, João Henrique Ferreira de Freitas
<joaohf@gmail.com> wrote:
> When extracting toolchain, if the list $executable_files has filenames
> with empty space character, the list will created but relocate_sdk.sh
> will not handle it well. This will lead to the below erro:
>
>     ./tmp/deploy/sdk/buildtools-mytools-x86_64-nativesdk-standalone-1.6.1.0.sh
>     Enter target directory for SDK (default: /opt/mydistro/mytoolset/1.6.1.0):
>     You are about to install the SDK to "/opt/mydistro/mytoolset/1.6.1.0". Proceed[Y/n]?
>     Extracting SDK...done
>     Setting it up.../opt/mydistro/mytoolset/1.6.1.0/relocate_sdk.sh: line 2: sintaxe error `token'  `('
>     /opt/mydistro/mytoolset/1.6.1.0/relocate_sdk.sh: line 2: `e
>
> The same occurs with replacement of ${SDKPATH} in configs/scripts/etc files.
>
> We should ensure that full path is protected before relocate_sdk.sh
> and ${SDKPATH} replacement calls.
>
> Signed-off-by: João Henrique Ferreira de Freitas <joaohf@gmail.com>

I am wondering if we cannot get rid of the 'tr' command that converts
newlines to '\0'. This would avoid a new fork of command. Did you
think about it?

-- 
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://code.ossystems.com.br
Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750

Re: [PATCH v2 1/2] python-distribute: add nativesdk to BBCLASSEXTEND

[Otavio Salvador]

On Mon, Jul 14, 2014 at 7:55 PM, João Henrique Ferreira de Freitas
<joaohf@gmail.com> wrote:
> Without this, python tools that need python-distribute doesn't install
> in SDK generated by OE.
>
> Signed-off-by: João Henrique Ferreira de Freitas <joaohf@gmail.com>
> ---
>  meta/recipes-devtools/python/python-distribute_0.6.32.bb | 8 +++++++-
>  1 file changed, 7 insertions(+), 1 deletion(-)
>
> diff --git a/meta/recipes-devtools/python/python-distribute_0.6.32.bb b/meta/recipes-devtools/python/python-distribute_0.6.32.bb
> index 58bb8b4..7000145 100644
> --- a/meta/recipes-devtools/python/python-distribute_0.6.32.bb
> +++ b/meta/recipes-devtools/python/python-distribute_0.6.32.bb
> @@ -39,6 +39,12 @@ do_install_append() {
>      echo "./${SRCNAME}-${PV}-py${PYTHON_BASEVERSION}.egg" > ${D}${PYTHON_SITEPACKAGES_DIR}/setuptools.pth
>  }
>
> +do_install_append_class-nativesdk() {
> +       sed -i -e '1s|^#!.*|#!/usr/bin/env python|' ${D}${PYTHON_SITEPACKAGES_DIR}/distribute-${PV}-py${PYTHON_BASEVERSION}.egg/setuptools/tests/test_resources.py
> +       sed -i -e '1s|^#!.*|#!/usr/bin/env python|' ${D}${bindir}/easy_install
> +       sed -i -e '1s|^#!.*|#!/usr/bin/env python|' ${D}${bindir}/easy_install-${PYTHON_BASEVERSION}
> +}

Couldn't this be done unconditionally for all target types?
"/usr/bin/env python" should work fine in target as well, no?

-- 
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://code.ossystems.com.br
Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750

Re: [PATCH v2 2/2] populate_sdk_base: ensure that filenames with empty space character are handled

[João Henrique Ferreira de Freitas]

Em 15-07-2014 10:33, Otavio Salvador escreveu:
> On Mon, Jul 14, 2014 at 7:55 PM, João Henrique Ferreira de Freitas
> <joaohf@gmail.com> wrote:
>> When extracting toolchain, if the list $executable_files has filenames
>> with empty space character, the list will created but relocate_sdk.sh
>> will not handle it well. This will lead to the below erro:
>>
>>      ./tmp/deploy/sdk/buildtools-mytools-x86_64-nativesdk-standalone-1.6.1.0.sh
>>      Enter target directory for SDK (default: /opt/mydistro/mytoolset/1.6.1.0):
>>      You are about to install the SDK to "/opt/mydistro/mytoolset/1.6.1.0". Proceed[Y/n]?
>>      Extracting SDK...done
>>      Setting it up.../opt/mydistro/mytoolset/1.6.1.0/relocate_sdk.sh: line 2: sintaxe error `token'  `('
>>      /opt/mydistro/mytoolset/1.6.1.0/relocate_sdk.sh: line 2: `e
>>
>> The same occurs with replacement of ${SDKPATH} in configs/scripts/etc files.
>>
>> We should ensure that full path is protected before relocate_sdk.sh
>> and ${SDKPATH} replacement calls.
>>
>> Signed-off-by: João Henrique Ferreira de Freitas <joaohf@gmail.com>
> I am wondering if we cannot get rid of the 'tr' command that converts
> newlines to '\0'. This would avoid a new fork of command. Did you
> think about it?
>
I think it's possible. Maybe  using: awk -F':' '{printf "%s\0", $1}' 
like this:

$SUDO_EXEC find $native_sysroot -type f -exec file '{}' \;|grep 
":.*\(ASCII\|script\|source\).*text"|awk -F':' '{printf "%s\0", 
$1}'|$SUDO_EXEC xargs -0 sed -i -e 
"s:$DEFAULT_INSTALL_DIR:$target_sdk_dir:g"

I will finish my tests before send a v3.

Thanks.

-- 
João Henrique Ferreira de Freitas - joaohf_at_gmail.com
Campinas-SP-Brasil

Re: [PATCH v2 1/2] python-distribute: add nativesdk to BBCLASSEXTEND

[João Henrique Ferreira de Freitas]

Hi Otavio,

Em 15-07-2014 10:35, Otavio Salvador escreveu:
> On Mon, Jul 14, 2014 at 7:55 PM, João Henrique Ferreira de Freitas
> <joaohf@gmail.com> wrote:
>> Without this, python tools that need python-distribute doesn't install
>> in SDK generated by OE.
>>
>> Signed-off-by: João Henrique Ferreira de Freitas <joaohf@gmail.com>
>> ---
>>   meta/recipes-devtools/python/python-distribute_0.6.32.bb | 8 +++++++-
>>   1 file changed, 7 insertions(+), 1 deletion(-)
>>
>> diff --git a/meta/recipes-devtools/python/python-distribute_0.6.32.bb b/meta/recipes-devtools/python/python-distribute_0.6.32.bb
>> index 58bb8b4..7000145 100644
>> --- a/meta/recipes-devtools/python/python-distribute_0.6.32.bb
>> +++ b/meta/recipes-devtools/python/python-distribute_0.6.32.bb
>> @@ -39,6 +39,12 @@ do_install_append() {
>>       echo "./${SRCNAME}-${PV}-py${PYTHON_BASEVERSION}.egg" > ${D}${PYTHON_SITEPACKAGES_DIR}/setuptools.pth
>>   }
>>
>> +do_install_append_class-nativesdk() {
>> +       sed -i -e '1s|^#!.*|#!/usr/bin/env python|' ${D}${PYTHON_SITEPACKAGES_DIR}/distribute-${PV}-py${PYTHON_BASEVERSION}.egg/setuptools/tests/test_resources.py
>> +       sed -i -e '1s|^#!.*|#!/usr/bin/env python|' ${D}${bindir}/easy_install
>> +       sed -i -e '1s|^#!.*|#!/usr/bin/env python|' ${D}${bindir}/easy_install-${PYTHON_BASEVERSION}
>> +}
> Couldn't this be done unconditionally for all target types?
> "/usr/bin/env python" should work fine in target as well, no?
>
I don't know if I fully understood your question...


Do you mean by 'all target types' like native, nativesdk ?

In some recipes (gnome-doc-utils.inc, createrepo_0.4.11.bb) , this 
pattern was used:

----
# Wrap the python script since the native python is
# ${bindir}/python-native/python, and the "#! /usr/bin/env python" can't
# find it since it is not in PATH.
do_install_append_class-native () {
  ...
   sed -i -e 's|^#!.*/usr/bin/env python|#! /usr/bin/env nativepython|' $i
  ...
}
----

Sometimes I feel really lost when need to fixes things like that. What 
is the correct way?

Without do_install_append_class-nativesdk() I got:

Computing transaction...error: Can't install nativesdk-python-distribute-0.6.32-ml5@x86_64_nativesdk: no package provides /usr/bin/python


Thanks.

-- 
João Henrique Ferreira de Freitas - joaohf_at_gmail.com
Campinas-SP-Brasil

Re: [PATCH v2 1/2] python-distribute: add nativesdk to BBCLASSEXTEND

[Otavio Salvador]

On Wed, Jul 16, 2014 at 9:50 PM, João Henrique Ferreira de Freitas
<joaohf@gmail.com> wrote:
> Em 15-07-2014 10:35, Otavio Salvador escreveu:
>> On Mon, Jul 14, 2014 at 7:55 PM, João Henrique Ferreira de Freitas
>> <joaohf@gmail.com> wrote:
>>>
>>> Without this, python tools that need python-distribute doesn't install
>>> in SDK generated by OE.
>>>
>>> Signed-off-by: João Henrique Ferreira de Freitas <joaohf@gmail.com>
>>> ---
>>>   meta/recipes-devtools/python/python-distribute_0.6.32.bb | 8 +++++++-
>>>   1 file changed, 7 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/meta/recipes-devtools/python/python-distribute_0.6.32.bb
>>> b/meta/recipes-devtools/python/python-distribute_0.6.32.bb
>>> index 58bb8b4..7000145 100644
>>> --- a/meta/recipes-devtools/python/python-distribute_0.6.32.bb
>>> +++ b/meta/recipes-devtools/python/python-distribute_0.6.32.bb
>>> @@ -39,6 +39,12 @@ do_install_append() {
>>>       echo "./${SRCNAME}-${PV}-py${PYTHON_BASEVERSION}.egg" >
>>> ${D}${PYTHON_SITEPACKAGES_DIR}/setuptools.pth
>>>   }
>>>
>>> +do_install_append_class-nativesdk() {
>>> +       sed -i -e '1s|^#!.*|#!/usr/bin/env python|'
>>> ${D}${PYTHON_SITEPACKAGES_DIR}/distribute-${PV}-py${PYTHON_BASEVERSION}.egg/setuptools/tests/test_resources.py
>>> +       sed -i -e '1s|^#!.*|#!/usr/bin/env python|'
>>> ${D}${bindir}/easy_install
>>> +       sed -i -e '1s|^#!.*|#!/usr/bin/env python|'
>>> ${D}${bindir}/easy_install-${PYTHON_BASEVERSION}
>>> +}
>>
>> Couldn't this be done unconditionally for all target types?
>> "/usr/bin/env python" should work fine in target as well, no?
>>
> I don't know if I fully understood your question...
>
>
> Do you mean by 'all target types' like native, nativesdk ?

Yes. To do it in do_install_append and change it for all targets ...

> In some recipes (gnome-doc-utils.inc, createrepo_0.4.11.bb) , this pattern
> was used:
>
> ----
> # Wrap the python script since the native python is
> # ${bindir}/python-native/python, and the "#! /usr/bin/env python" can't
> # find it since it is not in PATH.
> do_install_append_class-native () {
>  ...
>   sed -i -e 's|^#!.*/usr/bin/env python|#! /usr/bin/env nativepython|' $i
>  ...
> }
> ----
>
> Sometimes I feel really lost when need to fixes things like that. What is
> the correct way?

I think the above answer my question, it need to be nativepython for
native so it indeed must to be different.

> Without do_install_append_class-nativesdk() I got:
>
> Computing transaction...error: Can't install
> nativesdk-python-distribute-0.6.32-ml5@x86_64_nativesdk: no package provides
> /usr/bin/python

Your fix is fine than. Sorry for the noise :-)



-- 
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://code.ossystems.com.br
Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750

[PATCH v3 0/2] add nativesdk to BBCLASSEXTEND to python-distribute and to fix the side effect of it.

[João Henrique Ferreira de Freitas]

Changes in v3:

 - Use awk and remove two extra forks

João Henrique Ferreira de Freitas (2):
  python-distribute: add nativesdk to BBCLASSEXTEND
  populate_sdk_base: ensure that filenames with empty space character
    are handled

 meta/classes/populate_sdk_base.bbclass                   | 4 ++--
 meta/recipes-devtools/python/python-distribute_0.6.32.bb | 8 +++++++-
 2 files changed, 9 insertions(+), 3 deletions(-)

-- 
1.9.1

[PATCH v3 1/2] python-distribute: add nativesdk to BBCLASSEXTEND

[João Henrique Ferreira de Freitas]

Without this, python tools that need python-distribute doesn't install
in SDK generated by OE.

Signed-off-by: João Henrique Ferreira de Freitas <joaohf@gmail.com>
---
 meta/recipes-devtools/python/python-distribute_0.6.32.bb | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/meta/recipes-devtools/python/python-distribute_0.6.32.bb b/meta/recipes-devtools/python/python-distribute_0.6.32.bb
index 58bb8b4..7000145 100644
--- a/meta/recipes-devtools/python/python-distribute_0.6.32.bb
+++ b/meta/recipes-devtools/python/python-distribute_0.6.32.bb
@@ -39,6 +39,12 @@ do_install_append() {
     echo "./${SRCNAME}-${PV}-py${PYTHON_BASEVERSION}.egg" > ${D}${PYTHON_SITEPACKAGES_DIR}/setuptools.pth
 }
 
+do_install_append_class-nativesdk() {
+	sed -i -e '1s|^#!.*|#!/usr/bin/env python|' ${D}${PYTHON_SITEPACKAGES_DIR}/distribute-${PV}-py${PYTHON_BASEVERSION}.egg/setuptools/tests/test_resources.py
+	sed -i -e '1s|^#!.*|#!/usr/bin/env python|' ${D}${bindir}/easy_install
+	sed -i -e '1s|^#!.*|#!/usr/bin/env python|' ${D}${bindir}/easy_install-${PYTHON_BASEVERSION}
+}
+
 RDEPENDS_${PN} = "\
   python-distutils \
   python-compression \
@@ -46,4 +52,4 @@ RDEPENDS_${PN} = "\
 
 RPROVIDES_${PN} += "python-setuptools"
 
-BBCLASSEXTEND = "native"
+BBCLASSEXTEND = "native nativesdk"
-- 
1.9.1

[PATCH v3 2/2] populate_sdk_base: ensure that filenames with empty space character are handled

[João Henrique Ferreira de Freitas]

When extracting toolchain, if the list $executable_files has filenames
with empty space character, the list will created but relocate_sdk.sh
will not handle it well. This will lead to the below erro:

    ./tmp/deploy/sdk/buildtools-mytools-x86_64-nativesdk-standalone-1.6.1.0.sh
    Enter target directory for SDK (default: /opt/mydistro/mytoolset/1.6.1.0):
    You are about to install the SDK to "/opt/mydistro/mytoolset/1.6.1.0". Proceed[Y/n]?
    Extracting SDK...done
    Setting it up.../opt/mydistro/mytoolset/1.6.1.0/relocate_sdk.sh: line 2: sintaxe error `token'  `('
    /opt/mydistro/mytoolset/1.6.1.0/relocate_sdk.sh: line 2: `e

The same occurs with replacement of ${SDKPATH} in configs/scripts/etc files.

We should ensure that full path is protected before relocate_sdk.sh
and ${SDKPATH} replacement calls.

Signed-off-by: João Henrique Ferreira de Freitas <joaohf@gmail.com>
---
 meta/classes/populate_sdk_base.bbclass | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/meta/classes/populate_sdk_base.bbclass b/meta/classes/populate_sdk_base.bbclass
index a12bf11..0df98db 100644
--- a/meta/classes/populate_sdk_base.bbclass
+++ b/meta/classes/populate_sdk_base.bbclass
@@ -252,7 +252,7 @@ if [ "$dl_path" = "" ] ; then
 	echo "SDK could not be set up. Relocate script unable to find ld-linux.so. Abort!"
 	exit 1
 fi
-executable_files=$($SUDO_EXEC find $native_sysroot -type f -perm /111)
+executable_files=$($SUDO_EXEC find $native_sysroot -type f -perm /111 -printf "'%h/%f' ")
 
 tdir=`mktemp -d`
 if [ x$tdir = x ] ; then
@@ -273,7 +273,7 @@ if [ $relocate = 1 ] ; then
 fi
 
 # replace ${SDKPATH} with the new prefix in all text files: configs/scripts/etc
-$SUDO_EXEC find $native_sysroot -type f -exec file '{}' \;|grep ":.*\(ASCII\|script\|source\).*text"|cut -d':' -f1|$SUDO_EXEC xargs sed -i -e "s:$DEFAULT_INSTALL_DIR:$target_sdk_dir:g"
+$SUDO_EXEC find $native_sysroot -type f -exec file '{}' \;|grep ":.*\(ASCII\|script\|source\).*text"|awk -F':' '{printf "%s\0", $1}'|$SUDO_EXEC xargs -0 sed -i -e "s:$DEFAULT_INSTALL_DIR:$target_sdk_dir:g"
 
 # change all symlinks pointing to ${SDKPATH}
 for l in $($SUDO_EXEC find $native_sysroot -type l); do
-- 
1.9.1