Cross-compiling python fails with new custom tuning

Cross-compiling python fails with new custom tuning

[Michael Habibi]

[-- Attachment #1: Type: text/plain, Size: 4365 bytes --]

All,

I added a new machine definition with tuning parameters for haswell
microarchitectures (basically just duplicated corei7 but tuned it for
haswell). This seems to work correctly, but failed when running do_install
for python toward the end of the build process. I am running with the Yocto
2.0 framework, with very minimal changes to create a new distribution and
machine for our custom embedded device. Note I had this distro
configuration working before, and the only difference is I added a new
machine with this tuning.

I believe the issue is because, as part of the do_install step for Python,
it attempts to run python on the local host, despite being cross-compiled.
However, it is tuned for a processor that my host machine doesn't run, so I
get an instruction exception.

Did I do something weird? I figure python would be configured for
cross-compiling and therefore wouldn't try to run the target version on the
host, even for sanity tests. Here is the output of the failure:

$ tail -20
/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/temp/log.do_install.17258

x86_64-diags-linux-ar rc libpython2.7.a Modules/threadmodule.o
 Modules/signalmodule.o  Modules/posixmodule.o  Modules/errnomodule.o
 Modules/pwdmodule.o  Modules/_sre.o  Modules/_codecsmodule.o
 Modules/_weakref.o  Modules/zipimport.o  Modules/symtablemodule.o
 Modules/md5module.o Modules/md5.o  Modules/xxsubtype.o
x86_64-diags-linux-ranlib libpython2.7.a
Modules/posixmodule.o: In function `posix_tmpnam':
/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Modules/posixmodule.c:7575:
warning: the use of `tmpnam_r' is dangerous, better use `mkstemp'
Modules/posixmodule.o: In function `posix_tempnam':
/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Modules/posixmodule.c:7522:
warning: the use of `tempnam' is dangerous, better use `mkstemp'
x86_64-diags-linux-gcc  -m64 -march=haswell -mtune=haswell -mfpmath=sse
-mavx2 --sysroot=/projects/yocto-git/build/tmp/sysroots/continental -Wl,-O1
-Wl,--hash-style=gnu -Wl,--as-needed -Xlinker -export-dynamic -o python \
                        Modules/python.o \
                        -L. -lpython2.7 -lpthread -ldl  -lpthread -lutil
-lm
_PYTHON_PROJECT_BASE=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build
_PYTHON_HOST_PLATFORM=linux2-x86_64
PYTHONPATH=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build/build/lib.linux2-x86_64-2.7:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib/plat-linux2
python2.7 -S -m sysconfig --generate-posix-vars ;\
        if test $? -ne 0 ; then \
                echo "generate-posix-vars failed" ; \
                rm -f ./pybuilddir.txt ; \
                exit 1 ; \
        fi
Illegal instruction (core dumped)
make: *** [sharedmods] Error 132
WARNING: exit code 1 from a shell command.
ERROR: oe_runmake failed
ERROR: Function failed: do_install (log file is located at
/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/temp/log.do_install.17258)

Here is my tune-haswell.inc (ignore some copy/paste comment issues right
now =):

# Settings for the GCC(1) cpu-type "haswell":
#
#     Intel Core i7 CPU with 64-bit extensions, MOVBE, MMX, SSE, SSE2,
SSE3,·
#     SSSE3, SSE4.1, SSE4.2, POPCNT, AVX, AVX2, AES, PCLMUL, FSGSBASE,
#     RDRND, FMA, BMI, BMI2 and F16C instruction set support.
#
# This tune is recommended for Intel Nehalem and Silvermont (e.g. Bay
Trail) CPUs
# (and beyond).
#
DEFAULTTUNE ?= "haswell"

# Pull in the previous tune in to pull in PACKAGE_EXTRA_ARCHS
require conf/machine/include/tune-corei7.inc

# Extra tune features
TUNEVALID[haswell] = "Enable haswell specific processor optimizations"
TUNE_CCARGS .= "${@bb.utils.contains("TUNE_FEATURES", "haswell", "
-march=haswell -mtune=haswell -mfpmath=sse -mavx2", "", d)}"

# Extra tune selections
AVAILTUNES += "haswell"
TUNE_FEATURES_tune-haswell = "${TUNE_FEATURES_tune-x86-64} haswell"
BASE_LIB_tune-haswell = "lib"
TUNE_PKGARCH_tune-haswell = "haswell"
PACKAGE_EXTRA_ARCHS_tune-haswell = "${PACKAGE_EXTRA_ARCHS_tune-corei7}
haswell"

[-- Attachment #2: Type: text/html, Size: 5186 bytes --]

Re: Cross-compiling python fails with new custom tuning

[Mark Hatle]

My guess is that there is a bug in the python integration where it's not
realizing the host and target are different systems, so it's trying to run a
target program on your host.  Your host isn't haswell, so... Illegal Instruction
-- and the system stops.

The alternative would be something is running QEMU to execute a target binary
and QEMU doesn't have instruction support -- but that doesn't look like the case
here.

--Mark

On 11/24/15 3:06 PM, Michael Habibi wrote:
> All,
> 
> I added a new machine definition with tuning parameters for haswell
> microarchitectures (basically just duplicated corei7 but tuned it for haswell).
> This seems to work correctly, but failed when running do_install for python
> toward the end of the build process. I am running with the Yocto 2.0 framework,
> with very minimal changes to create a new distribution and machine for our
> custom embedded device. Note I had this distro configuration working before, and
> the only difference is I added a new machine with this tuning.
> 
> I believe the issue is because, as part of the do_install step for Python, it
> attempts to run python on the local host, despite being cross-compiled. However,
> it is tuned for a processor that my host machine doesn't run, so I get an
> instruction exception.
> 
> Did I do something weird? I figure python would be configured for
> cross-compiling and therefore wouldn't try to run the target version on the
> host, even for sanity tests. Here is the output of the failure:
> 
> $ tail -20
> /projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/temp/log.do_install.17258
> 
> x86_64-diags-linux-ar rc libpython2.7.a Modules/threadmodule.o
>  Modules/signalmodule.o  Modules/posixmodule.o  Modules/errnomodule.o
>  Modules/pwdmodule.o  Modules/_sre.o  Modules/_codecsmodule.o
>  Modules/_weakref.o  Modules/zipimport.o  Modules/symtablemodule.o
>  Modules/md5module.o Modules/md5.o  Modules/xxsubtype.o
> x86_64-diags-linux-ranlib libpython2.7.a
> Modules/posixmodule.o: In function `posix_tmpnam':
> /projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Modules/posixmodule.c:7575:
> warning: the use of `tmpnam_r' is dangerous, better use `mkstemp'
> Modules/posixmodule.o: In function `posix_tempnam':
> /projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Modules/posixmodule.c:7522:
> warning: the use of `tempnam' is dangerous, better use `mkstemp'
> x86_64-diags-linux-gcc  -m64 -march=haswell -mtune=haswell -mfpmath=sse -mavx2
> --sysroot=/projects/yocto-git/build/tmp/sysroots/continental -Wl,-O1
> -Wl,--hash-style=gnu -Wl,--as-needed -Xlinker -export-dynamic -o python \
>                         Modules/python.o \
>                         -L. -lpython2.7 -lpthread -ldl  -lpthread -lutil   -lm
> _PYTHON_PROJECT_BASE=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build
> _PYTHON_HOST_PLATFORM=linux2-x86_64
> PYTHONPATH=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build/build/lib.linux2-x86_64-2.7:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib/plat-linux2
> python2.7 -S -m sysconfig --generate-posix-vars ;\
>         if test $? -ne 0 ; then \
>                 echo "generate-posix-vars failed" ; \
>                 rm -f ./pybuilddir.txt ; \
>                 exit 1 ; \
>         fi
> Illegal instruction (core dumped)
> make: *** [sharedmods] Error 132
> WARNING: exit code 1 from a shell command.
> ERROR: oe_runmake failed
> ERROR: Function failed: do_install (log file is located at
> /projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/temp/log.do_install.17258)
> 
> Here is my tune-haswell.inc (ignore some copy/paste comment issues right now =):
> 
> # Settings for the GCC(1) cpu-type "haswell":
> #
> #     Intel Core i7 CPU with 64-bit extensions, MOVBE, MMX, SSE, SSE2, SSE3,·
> #     SSSE3, SSE4.1, SSE4.2, POPCNT, AVX, AVX2, AES, PCLMUL, FSGSBASE,
> #     RDRND, FMA, BMI, BMI2 and F16C instruction set support.
> #
> # This tune is recommended for Intel Nehalem and Silvermont (e.g. Bay Trail) CPUs
> # (and beyond).
> #
> DEFAULTTUNE ?= "haswell"
> 
> # Pull in the previous tune in to pull in PACKAGE_EXTRA_ARCHS
> require conf/machine/include/tune-corei7.inc
> 
> # Extra tune features
> TUNEVALID[haswell] = "Enable haswell specific processor optimizations"
> TUNE_CCARGS .= "${@bb.utils.contains("TUNE_FEATURES", "haswell", "
> -march=haswell -mtune=haswell -mfpmath=sse -mavx2", "", d)}"
> 
> # Extra tune selections
> AVAILTUNES += "haswell"
> TUNE_FEATURES_tune-haswell = "${TUNE_FEATURES_tune-x86-64} haswell"
> BASE_LIB_tune-haswell = "lib"
> TUNE_PKGARCH_tune-haswell = "haswell"
> PACKAGE_EXTRA_ARCHS_tune-haswell = "${PACKAGE_EXTRA_ARCHS_tune-corei7} haswell"
> 
> 
> 

Re: Cross-compiling python fails with new custom tuning

[Mark Hatle]

On 11/24/15 3:23 PM, Mark Hatle wrote:
> My guess is that there is a bug in the python integration where it's not
> realizing the host and target are different systems, so it's trying to run a
> target program on your host.  Your host isn't haswell, so... Illegal Instruction
> -- and the system stops.

Just an FYI, I hit the same problem today.  I suspect it is the host trying to
run target software and I'm looking into it.

--Mark

> The alternative would be something is running QEMU to execute a target binary
> and QEMU doesn't have instruction support -- but that doesn't look like the case
> here.
> 
> --Mark
> 
> On 11/24/15 3:06 PM, Michael Habibi wrote:
>> All,
>>
>> I added a new machine definition with tuning parameters for haswell
>> microarchitectures (basically just duplicated corei7 but tuned it for haswell).
>> This seems to work correctly, but failed when running do_install for python
>> toward the end of the build process. I am running with the Yocto 2.0 framework,
>> with very minimal changes to create a new distribution and machine for our
>> custom embedded device. Note I had this distro configuration working before, and
>> the only difference is I added a new machine with this tuning.
>>
>> I believe the issue is because, as part of the do_install step for Python, it
>> attempts to run python on the local host, despite being cross-compiled. However,
>> it is tuned for a processor that my host machine doesn't run, so I get an
>> instruction exception.
>>
>> Did I do something weird? I figure python would be configured for
>> cross-compiling and therefore wouldn't try to run the target version on the
>> host, even for sanity tests. Here is the output of the failure:
>>
>> $ tail -20
>> /projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/temp/log.do_install.17258
>>
>> x86_64-diags-linux-ar rc libpython2.7.a Modules/threadmodule.o
>>  Modules/signalmodule.o  Modules/posixmodule.o  Modules/errnomodule.o
>>  Modules/pwdmodule.o  Modules/_sre.o  Modules/_codecsmodule.o
>>  Modules/_weakref.o  Modules/zipimport.o  Modules/symtablemodule.o
>>  Modules/md5module.o Modules/md5.o  Modules/xxsubtype.o
>> x86_64-diags-linux-ranlib libpython2.7.a
>> Modules/posixmodule.o: In function `posix_tmpnam':
>> /projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Modules/posixmodule.c:7575:
>> warning: the use of `tmpnam_r' is dangerous, better use `mkstemp'
>> Modules/posixmodule.o: In function `posix_tempnam':
>> /projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Modules/posixmodule.c:7522:
>> warning: the use of `tempnam' is dangerous, better use `mkstemp'
>> x86_64-diags-linux-gcc  -m64 -march=haswell -mtune=haswell -mfpmath=sse -mavx2
>> --sysroot=/projects/yocto-git/build/tmp/sysroots/continental -Wl,-O1
>> -Wl,--hash-style=gnu -Wl,--as-needed -Xlinker -export-dynamic -o python \
>>                         Modules/python.o \
>>                         -L. -lpython2.7 -lpthread -ldl  -lpthread -lutil   -lm
>> _PYTHON_PROJECT_BASE=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build
>> _PYTHON_HOST_PLATFORM=linux2-x86_64
>> PYTHONPATH=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build/build/lib.linux2-x86_64-2.7:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib/plat-linux2
>> python2.7 -S -m sysconfig --generate-posix-vars ;\
>>         if test $? -ne 0 ; then \
>>                 echo "generate-posix-vars failed" ; \
>>                 rm -f ./pybuilddir.txt ; \
>>                 exit 1 ; \
>>         fi
>> Illegal instruction (core dumped)
>> make: *** [sharedmods] Error 132
>> WARNING: exit code 1 from a shell command.
>> ERROR: oe_runmake failed
>> ERROR: Function failed: do_install (log file is located at
>> /projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/temp/log.do_install.17258)
>>
>> Here is my tune-haswell.inc (ignore some copy/paste comment issues right now =):
>>
>> # Settings for the GCC(1) cpu-type "haswell":
>> #
>> #     Intel Core i7 CPU with 64-bit extensions, MOVBE, MMX, SSE, SSE2, SSE3,·
>> #     SSSE3, SSE4.1, SSE4.2, POPCNT, AVX, AVX2, AES, PCLMUL, FSGSBASE,
>> #     RDRND, FMA, BMI, BMI2 and F16C instruction set support.
>> #
>> # This tune is recommended for Intel Nehalem and Silvermont (e.g. Bay Trail) CPUs
>> # (and beyond).
>> #
>> DEFAULTTUNE ?= "haswell"
>>
>> # Pull in the previous tune in to pull in PACKAGE_EXTRA_ARCHS
>> require conf/machine/include/tune-corei7.inc
>>
>> # Extra tune features
>> TUNEVALID[haswell] = "Enable haswell specific processor optimizations"
>> TUNE_CCARGS .= "${@bb.utils.contains("TUNE_FEATURES", "haswell", "
>> -march=haswell -mtune=haswell -mfpmath=sse -mavx2", "", d)}"
>>
>> # Extra tune selections
>> AVAILTUNES += "haswell"
>> TUNE_FEATURES_tune-haswell = "${TUNE_FEATURES_tune-x86-64} haswell"
>> BASE_LIB_tune-haswell = "lib"
>> TUNE_PKGARCH_tune-haswell = "haswell"
>> PACKAGE_EXTRA_ARCHS_tune-haswell = "${PACKAGE_EXTRA_ARCHS_tune-corei7} haswell"
>>
>>
>>
> 

Re: Cross-compiling python fails with new custom tuning

[Michael Habibi]

[-- Attachment #1: Type: text/plain, Size: 6963 bytes --]

For what it's worth, this is the offending portion of Makefile.pre:452:

# Create build directory and generate the sysconfig build-time data there.
# pybuilddir.txt contains the name of the build dir and is used for
# sys.path fixup -- see Modules/getpath.c.
# Since this step runs before shared modules are built, try to avoid
bootstrap
# problems by creating a dummy pybuilddir.txt just to allow interpreter
# initialization to succeed.  It will be overwritten by generate-posix-vars
# or removed in case of failure.
pybuilddir.txt: $(BUILDPYTHON)
→ @echo "none" > ./pybuilddir.txt
→ $(RUNSHARED) $(PYTHON_FOR_BUILD) -S -m sysconfig --generate-posix-vars ;\
→ if test $$? -ne 0 ; then \
→ → echo "generate-posix-vars failed" ; \
→ → rm -f ./pybuilddir.txt ; \
→ → exit 1 ; \
→ fi

I don't know enough about the Python build to understand what it's trying
to do, but perhaps replacing PYTHON_FOR_BUILD with HOSTPYTHON may be
acceptable?

I'm surprised this seems to work for other people, since this should be
failing for anyone using Python on a target platform different from their
host.

On Wed, Nov 25, 2015 at 9:02 AM, Mark Hatle <mark.hatle@windriver.com>
wrote:

> On 11/24/15 3:23 PM, Mark Hatle wrote:
> > My guess is that there is a bug in the python integration where it's not
> > realizing the host and target are different systems, so it's trying to
> run a
> > target program on your host.  Your host isn't haswell, so... Illegal
> Instruction
> > -- and the system stops.
>
> Just an FYI, I hit the same problem today.  I suspect it is the host
> trying to
> run target software and I'm looking into it.
>
> --Mark
>
> > The alternative would be something is running QEMU to execute a target
> binary
> > and QEMU doesn't have instruction support -- but that doesn't look like
> the case
> > here.
> >
> > --Mark
> >
> > On 11/24/15 3:06 PM, Michael Habibi wrote:
> >> All,
> >>
> >> I added a new machine definition with tuning parameters for haswell
> >> microarchitectures (basically just duplicated corei7 but tuned it for
> haswell).
> >> This seems to work correctly, but failed when running do_install for
> python
> >> toward the end of the build process. I am running with the Yocto 2.0
> framework,
> >> with very minimal changes to create a new distribution and machine for
> our
> >> custom embedded device. Note I had this distro configuration working
> before, and
> >> the only difference is I added a new machine with this tuning.
> >>
> >> I believe the issue is because, as part of the do_install step for
> Python, it
> >> attempts to run python on the local host, despite being cross-compiled.
> However,
> >> it is tuned for a processor that my host machine doesn't run, so I get
> an
> >> instruction exception.
> >>
> >> Did I do something weird? I figure python would be configured for
> >> cross-compiling and therefore wouldn't try to run the target version on
> the
> >> host, even for sanity tests. Here is the output of the failure:
> >>
> >> $ tail -20
> >>
> /projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/temp/log.do_install.17258
> >>
> >> x86_64-diags-linux-ar rc libpython2.7.a Modules/threadmodule.o
> >>  Modules/signalmodule.o  Modules/posixmodule.o  Modules/errnomodule.o
> >>  Modules/pwdmodule.o  Modules/_sre.o  Modules/_codecsmodule.o
> >>  Modules/_weakref.o  Modules/zipimport.o  Modules/symtablemodule.o
> >>  Modules/md5module.o Modules/md5.o  Modules/xxsubtype.o
> >> x86_64-diags-linux-ranlib libpython2.7.a
> >> Modules/posixmodule.o: In function `posix_tmpnam':
> >>
> /projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Modules/posixmodule.c:7575:
> >> warning: the use of `tmpnam_r' is dangerous, better use `mkstemp'
> >> Modules/posixmodule.o: In function `posix_tempnam':
> >>
> /projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Modules/posixmodule.c:7522:
> >> warning: the use of `tempnam' is dangerous, better use `mkstemp'
> >> x86_64-diags-linux-gcc  -m64 -march=haswell -mtune=haswell -mfpmath=sse
> -mavx2
> >> --sysroot=/projects/yocto-git/build/tmp/sysroots/continental -Wl,-O1
> >> -Wl,--hash-style=gnu -Wl,--as-needed -Xlinker -export-dynamic -o python
> \
> >>                         Modules/python.o \
> >>                         -L. -lpython2.7 -lpthread -ldl  -lpthread
> -lutil   -lm
> >>
> _PYTHON_PROJECT_BASE=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build
> >> _PYTHON_HOST_PLATFORM=linux2-x86_64
> >>
> PYTHONPATH=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build/build/lib.linux2-x86_64-2.7:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib/plat-linux2
> >> python2.7 -S -m sysconfig --generate-posix-vars ;\
> >>         if test $? -ne 0 ; then \
> >>                 echo "generate-posix-vars failed" ; \
> >>                 rm -f ./pybuilddir.txt ; \
> >>                 exit 1 ; \
> >>         fi
> >> Illegal instruction (core dumped)
> >> make: *** [sharedmods] Error 132
> >> WARNING: exit code 1 from a shell command.
> >> ERROR: oe_runmake failed
> >> ERROR: Function failed: do_install (log file is located at
> >>
> /projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/temp/log.do_install.17258)
> >>
> >> Here is my tune-haswell.inc (ignore some copy/paste comment issues
> right now =):
> >>
> >> # Settings for the GCC(1) cpu-type "haswell":
> >> #
> >> #     Intel Core i7 CPU with 64-bit extensions, MOVBE, MMX, SSE, SSE2,
> SSE3,·
> >> #     SSSE3, SSE4.1, SSE4.2, POPCNT, AVX, AVX2, AES, PCLMUL, FSGSBASE,
> >> #     RDRND, FMA, BMI, BMI2 and F16C instruction set support.
> >> #
> >> # This tune is recommended for Intel Nehalem and Silvermont (e.g. Bay
> Trail) CPUs
> >> # (and beyond).
> >> #
> >> DEFAULTTUNE ?= "haswell"
> >>
> >> # Pull in the previous tune in to pull in PACKAGE_EXTRA_ARCHS
> >> require conf/machine/include/tune-corei7.inc
> >>
> >> # Extra tune features
> >> TUNEVALID[haswell] = "Enable haswell specific processor optimizations"
> >> TUNE_CCARGS .= "${@bb.utils.contains("TUNE_FEATURES", "haswell", "
> >> -march=haswell -mtune=haswell -mfpmath=sse -mavx2", "", d)}"
> >>
> >> # Extra tune selections
> >> AVAILTUNES += "haswell"
> >> TUNE_FEATURES_tune-haswell = "${TUNE_FEATURES_tune-x86-64} haswell"
> >> BASE_LIB_tune-haswell = "lib"
> >> TUNE_PKGARCH_tune-haswell = "haswell"
> >> PACKAGE_EXTRA_ARCHS_tune-haswell = "${PACKAGE_EXTRA_ARCHS_tune-corei7}
> haswell"
> >>
> >>
> >>
> >
>
> --
> _______________________________________________
> yocto mailing list
> yocto@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/yocto
>

[-- Attachment #2: Type: text/html, Size: 8708 bytes --]

Re: Cross-compiling python fails with new custom tuning

[Michael Habibi]

[-- Attachment #1: Type: text/plain, Size: 7625 bytes --]

Ah sorry, I misspoke. I walked through the Makefile and configure scripts a
bit more, and $(PYTHON_FOR_BUILD) should in fact be a host version of
Python. It seems it's not being configured correctly. Either the path is
setup wrong and python2.7 is not pointing to the right version, or
python2.7 needs to be pointing to the absolute path instead of relying on
$PATH.




On Wed, Nov 25, 2015 at 9:45 AM, Michael Habibi <mikehabibi@gmail.com>
wrote:

> For what it's worth, this is the offending portion of Makefile.pre:452:
>
> # Create build directory and generate the sysconfig build-time data there.
> # pybuilddir.txt contains the name of the build dir and is used for
> # sys.path fixup -- see Modules/getpath.c.
> # Since this step runs before shared modules are built, try to avoid
> bootstrap
> # problems by creating a dummy pybuilddir.txt just to allow interpreter
> # initialization to succeed.  It will be overwritten by generate-posix-vars
> # or removed in case of failure.
> pybuilddir.txt: $(BUILDPYTHON)
> → @echo "none" > ./pybuilddir.txt
> → $(RUNSHARED) $(PYTHON_FOR_BUILD) -S -m sysconfig --generate-posix-vars ;\
> → if test $$? -ne 0 ; then \
> → → echo "generate-posix-vars failed" ; \
> → → rm -f ./pybuilddir.txt ; \
> → → exit 1 ; \
> → fi
>
> I don't know enough about the Python build to understand what it's trying
> to do, but perhaps replacing PYTHON_FOR_BUILD with HOSTPYTHON may be
> acceptable?
>
> I'm surprised this seems to work for other people, since this should be
> failing for anyone using Python on a target platform different from their
> host.
>
> On Wed, Nov 25, 2015 at 9:02 AM, Mark Hatle <mark.hatle@windriver.com>
> wrote:
>
>> On 11/24/15 3:23 PM, Mark Hatle wrote:
>> > My guess is that there is a bug in the python integration where it's not
>> > realizing the host and target are different systems, so it's trying to
>> run a
>> > target program on your host.  Your host isn't haswell, so... Illegal
>> Instruction
>> > -- and the system stops.
>>
>> Just an FYI, I hit the same problem today.  I suspect it is the host
>> trying to
>> run target software and I'm looking into it.
>>
>> --Mark
>>
>> > The alternative would be something is running QEMU to execute a target
>> binary
>> > and QEMU doesn't have instruction support -- but that doesn't look like
>> the case
>> > here.
>> >
>> > --Mark
>> >
>> > On 11/24/15 3:06 PM, Michael Habibi wrote:
>> >> All,
>> >>
>> >> I added a new machine definition with tuning parameters for haswell
>> >> microarchitectures (basically just duplicated corei7 but tuned it for
>> haswell).
>> >> This seems to work correctly, but failed when running do_install for
>> python
>> >> toward the end of the build process. I am running with the Yocto 2.0
>> framework,
>> >> with very minimal changes to create a new distribution and machine for
>> our
>> >> custom embedded device. Note I had this distro configuration working
>> before, and
>> >> the only difference is I added a new machine with this tuning.
>> >>
>> >> I believe the issue is because, as part of the do_install step for
>> Python, it
>> >> attempts to run python on the local host, despite being
>> cross-compiled. However,
>> >> it is tuned for a processor that my host machine doesn't run, so I get
>> an
>> >> instruction exception.
>> >>
>> >> Did I do something weird? I figure python would be configured for
>> >> cross-compiling and therefore wouldn't try to run the target version
>> on the
>> >> host, even for sanity tests. Here is the output of the failure:
>> >>
>> >> $ tail -20
>> >>
>> /projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/temp/log.do_install.17258
>> >>
>> >> x86_64-diags-linux-ar rc libpython2.7.a Modules/threadmodule.o
>> >>  Modules/signalmodule.o  Modules/posixmodule.o  Modules/errnomodule.o
>> >>  Modules/pwdmodule.o  Modules/_sre.o  Modules/_codecsmodule.o
>> >>  Modules/_weakref.o  Modules/zipimport.o  Modules/symtablemodule.o
>> >>  Modules/md5module.o Modules/md5.o  Modules/xxsubtype.o
>> >> x86_64-diags-linux-ranlib libpython2.7.a
>> >> Modules/posixmodule.o: In function `posix_tmpnam':
>> >>
>> /projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Modules/posixmodule.c:7575:
>> >> warning: the use of `tmpnam_r' is dangerous, better use `mkstemp'
>> >> Modules/posixmodule.o: In function `posix_tempnam':
>> >>
>> /projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Modules/posixmodule.c:7522:
>> >> warning: the use of `tempnam' is dangerous, better use `mkstemp'
>> >> x86_64-diags-linux-gcc  -m64 -march=haswell -mtune=haswell
>> -mfpmath=sse -mavx2
>> >> --sysroot=/projects/yocto-git/build/tmp/sysroots/continental -Wl,-O1
>> >> -Wl,--hash-style=gnu -Wl,--as-needed -Xlinker -export-dynamic -o
>> python \
>> >>                         Modules/python.o \
>> >>                         -L. -lpython2.7 -lpthread -ldl  -lpthread
>> -lutil   -lm
>> >>
>> _PYTHON_PROJECT_BASE=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build
>> >> _PYTHON_HOST_PLATFORM=linux2-x86_64
>> >>
>> PYTHONPATH=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build/build/lib.linux2-x86_64-2.7:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib/plat-linux2
>> >> python2.7 -S -m sysconfig --generate-posix-vars ;\
>> >>         if test $? -ne 0 ; then \
>> >>                 echo "generate-posix-vars failed" ; \
>> >>                 rm -f ./pybuilddir.txt ; \
>> >>                 exit 1 ; \
>> >>         fi
>> >> Illegal instruction (core dumped)
>> >> make: *** [sharedmods] Error 132
>> >> WARNING: exit code 1 from a shell command.
>> >> ERROR: oe_runmake failed
>> >> ERROR: Function failed: do_install (log file is located at
>> >>
>> /projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/temp/log.do_install.17258)
>> >>
>> >> Here is my tune-haswell.inc (ignore some copy/paste comment issues
>> right now =):
>> >>
>> >> # Settings for the GCC(1) cpu-type "haswell":
>> >> #
>> >> #     Intel Core i7 CPU with 64-bit extensions, MOVBE, MMX, SSE, SSE2,
>> SSE3,·
>> >> #     SSSE3, SSE4.1, SSE4.2, POPCNT, AVX, AVX2, AES, PCLMUL, FSGSBASE,
>> >> #     RDRND, FMA, BMI, BMI2 and F16C instruction set support.
>> >> #
>> >> # This tune is recommended for Intel Nehalem and Silvermont (e.g. Bay
>> Trail) CPUs
>> >> # (and beyond).
>> >> #
>> >> DEFAULTTUNE ?= "haswell"
>> >>
>> >> # Pull in the previous tune in to pull in PACKAGE_EXTRA_ARCHS
>> >> require conf/machine/include/tune-corei7.inc
>> >>
>> >> # Extra tune features
>> >> TUNEVALID[haswell] = "Enable haswell specific processor optimizations"
>> >> TUNE_CCARGS .= "${@bb.utils.contains("TUNE_FEATURES", "haswell", "
>> >> -march=haswell -mtune=haswell -mfpmath=sse -mavx2", "", d)}"
>> >>
>> >> # Extra tune selections
>> >> AVAILTUNES += "haswell"
>> >> TUNE_FEATURES_tune-haswell = "${TUNE_FEATURES_tune-x86-64} haswell"
>> >> BASE_LIB_tune-haswell = "lib"
>> >> TUNE_PKGARCH_tune-haswell = "haswell"
>> >> PACKAGE_EXTRA_ARCHS_tune-haswell = "${PACKAGE_EXTRA_ARCHS_tune-corei7}
>> haswell"
>> >>
>> >>
>> >>
>> >
>>
>> --
>> _______________________________________________
>> yocto mailing list
>> yocto@yoctoproject.org
>> https://lists.yoctoproject.org/listinfo/yocto
>>
>
>

[-- Attachment #2: Type: text/html, Size: 9555 bytes --]

Re: Cross-compiling python fails with new custom tuning

[Michael Habibi]

[-- Attachment #1: Type: text/plain, Size: 9104 bytes --]

Well, I'm wrong yet again. A 'which python2.7' shows that it's pointing to
a native version of Python:

*| which python2.7*
*|
/projects/yocto-git/build/tmp/sysroots/x86_64-linux/usr/bin/python-native/python2.7*
|
_PYTHON_PROJECT_BASE=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build
_PYTHON_HOST_PLATFORM=linux2-x86_64
PYTHONPATH=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build/build/lib.linux2-x86_64-2.7:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib/plat-linux2
python2.7 -S -m sysconfig --generate-posix-vars ;\
|       if test $? -ne 0 ; then \
|               echo "generate-posix-vars failed" ; \
|               rm -f ./pybuilddir.txt ; \
|               exit 1 ; \
|       fi
| Illegal instruction (core dumped)
| make: *** [sharedmods] Error 132

I guess one of the environmental variables being passed in is screwing up
what it's doing. Unfortunately I don't know enough about the inner workings
of Python to be of much help moving forward. I gave it my best shot!


On Wed, Nov 25, 2015 at 10:01 AM, Michael Habibi <mikehabibi@gmail.com>
wrote:

> Ah sorry, I misspoke. I walked through the Makefile and configure scripts
> a bit more, and $(PYTHON_FOR_BUILD) should in fact be a host version of
> Python. It seems it's not being configured correctly. Either the path is
> setup wrong and python2.7 is not pointing to the right version, or
> python2.7 needs to be pointing to the absolute path instead of relying on
> $PATH.
>
>
>
>
> On Wed, Nov 25, 2015 at 9:45 AM, Michael Habibi <mikehabibi@gmail.com>
> wrote:
>
>> For what it's worth, this is the offending portion of Makefile.pre:452:
>>
>> # Create build directory and generate the sysconfig build-time data there.
>> # pybuilddir.txt contains the name of the build dir and is used for
>> # sys.path fixup -- see Modules/getpath.c.
>> # Since this step runs before shared modules are built, try to avoid
>> bootstrap
>> # problems by creating a dummy pybuilddir.txt just to allow interpreter
>> # initialization to succeed.  It will be overwritten by
>> generate-posix-vars
>> # or removed in case of failure.
>> pybuilddir.txt: $(BUILDPYTHON)
>> → @echo "none" > ./pybuilddir.txt
>> → $(RUNSHARED) $(PYTHON_FOR_BUILD) -S -m sysconfig --generate-posix-vars
>> ;\
>> → if test $$? -ne 0 ; then \
>> → → echo "generate-posix-vars failed" ; \
>> → → rm -f ./pybuilddir.txt ; \
>> → → exit 1 ; \
>> → fi
>>
>> I don't know enough about the Python build to understand what it's trying
>> to do, but perhaps replacing PYTHON_FOR_BUILD with HOSTPYTHON may be
>> acceptable?
>>
>> I'm surprised this seems to work for other people, since this should be
>> failing for anyone using Python on a target platform different from their
>> host.
>>
>> On Wed, Nov 25, 2015 at 9:02 AM, Mark Hatle <mark.hatle@windriver.com>
>> wrote:
>>
>>> On 11/24/15 3:23 PM, Mark Hatle wrote:
>>> > My guess is that there is a bug in the python integration where it's
>>> not
>>> > realizing the host and target are different systems, so it's trying to
>>> run a
>>> > target program on your host.  Your host isn't haswell, so... Illegal
>>> Instruction
>>> > -- and the system stops.
>>>
>>> Just an FYI, I hit the same problem today.  I suspect it is the host
>>> trying to
>>> run target software and I'm looking into it.
>>>
>>> --Mark
>>>
>>> > The alternative would be something is running QEMU to execute a target
>>> binary
>>> > and QEMU doesn't have instruction support -- but that doesn't look
>>> like the case
>>> > here.
>>> >
>>> > --Mark
>>> >
>>> > On 11/24/15 3:06 PM, Michael Habibi wrote:
>>> >> All,
>>> >>
>>> >> I added a new machine definition with tuning parameters for haswell
>>> >> microarchitectures (basically just duplicated corei7 but tuned it for
>>> haswell).
>>> >> This seems to work correctly, but failed when running do_install for
>>> python
>>> >> toward the end of the build process. I am running with the Yocto 2.0
>>> framework,
>>> >> with very minimal changes to create a new distribution and machine
>>> for our
>>> >> custom embedded device. Note I had this distro configuration working
>>> before, and
>>> >> the only difference is I added a new machine with this tuning.
>>> >>
>>> >> I believe the issue is because, as part of the do_install step for
>>> Python, it
>>> >> attempts to run python on the local host, despite being
>>> cross-compiled. However,
>>> >> it is tuned for a processor that my host machine doesn't run, so I
>>> get an
>>> >> instruction exception.
>>> >>
>>> >> Did I do something weird? I figure python would be configured for
>>> >> cross-compiling and therefore wouldn't try to run the target version
>>> on the
>>> >> host, even for sanity tests. Here is the output of the failure:
>>> >>
>>> >> $ tail -20
>>> >>
>>> /projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/temp/log.do_install.17258
>>> >>
>>> >> x86_64-diags-linux-ar rc libpython2.7.a Modules/threadmodule.o
>>> >>  Modules/signalmodule.o  Modules/posixmodule.o  Modules/errnomodule.o
>>> >>  Modules/pwdmodule.o  Modules/_sre.o  Modules/_codecsmodule.o
>>> >>  Modules/_weakref.o  Modules/zipimport.o  Modules/symtablemodule.o
>>> >>  Modules/md5module.o Modules/md5.o  Modules/xxsubtype.o
>>> >> x86_64-diags-linux-ranlib libpython2.7.a
>>> >> Modules/posixmodule.o: In function `posix_tmpnam':
>>> >>
>>> /projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Modules/posixmodule.c:7575:
>>> >> warning: the use of `tmpnam_r' is dangerous, better use `mkstemp'
>>> >> Modules/posixmodule.o: In function `posix_tempnam':
>>> >>
>>> /projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Modules/posixmodule.c:7522:
>>> >> warning: the use of `tempnam' is dangerous, better use `mkstemp'
>>> >> x86_64-diags-linux-gcc  -m64 -march=haswell -mtune=haswell
>>> -mfpmath=sse -mavx2
>>> >> --sysroot=/projects/yocto-git/build/tmp/sysroots/continental -Wl,-O1
>>> >> -Wl,--hash-style=gnu -Wl,--as-needed -Xlinker -export-dynamic -o
>>> python \
>>> >>                         Modules/python.o \
>>> >>                         -L. -lpython2.7 -lpthread -ldl  -lpthread
>>> -lutil   -lm
>>> >>
>>> _PYTHON_PROJECT_BASE=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build
>>> >> _PYTHON_HOST_PLATFORM=linux2-x86_64
>>> >>
>>> PYTHONPATH=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build/build/lib.linux2-x86_64-2.7:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib/plat-linux2
>>> >> python2.7 -S -m sysconfig --generate-posix-vars ;\
>>> >>         if test $? -ne 0 ; then \
>>> >>                 echo "generate-posix-vars failed" ; \
>>> >>                 rm -f ./pybuilddir.txt ; \
>>> >>                 exit 1 ; \
>>> >>         fi
>>> >> Illegal instruction (core dumped)
>>> >> make: *** [sharedmods] Error 132
>>> >> WARNING: exit code 1 from a shell command.
>>> >> ERROR: oe_runmake failed
>>> >> ERROR: Function failed: do_install (log file is located at
>>> >>
>>> /projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/temp/log.do_install.17258)
>>> >>
>>> >> Here is my tune-haswell.inc (ignore some copy/paste comment issues
>>> right now =):
>>> >>
>>> >> # Settings for the GCC(1) cpu-type "haswell":
>>> >> #
>>> >> #     Intel Core i7 CPU with 64-bit extensions, MOVBE, MMX, SSE,
>>> SSE2, SSE3,·
>>> >> #     SSSE3, SSE4.1, SSE4.2, POPCNT, AVX, AVX2, AES, PCLMUL, FSGSBASE,
>>> >> #     RDRND, FMA, BMI, BMI2 and F16C instruction set support.
>>> >> #
>>> >> # This tune is recommended for Intel Nehalem and Silvermont (e.g. Bay
>>> Trail) CPUs
>>> >> # (and beyond).
>>> >> #
>>> >> DEFAULTTUNE ?= "haswell"
>>> >>
>>> >> # Pull in the previous tune in to pull in PACKAGE_EXTRA_ARCHS
>>> >> require conf/machine/include/tune-corei7.inc
>>> >>
>>> >> # Extra tune features
>>> >> TUNEVALID[haswell] = "Enable haswell specific processor optimizations"
>>> >> TUNE_CCARGS .= "${@bb.utils.contains("TUNE_FEATURES", "haswell", "
>>> >> -march=haswell -mtune=haswell -mfpmath=sse -mavx2", "", d)}"
>>> >>
>>> >> # Extra tune selections
>>> >> AVAILTUNES += "haswell"
>>> >> TUNE_FEATURES_tune-haswell = "${TUNE_FEATURES_tune-x86-64} haswell"
>>> >> BASE_LIB_tune-haswell = "lib"
>>> >> TUNE_PKGARCH_tune-haswell = "haswell"
>>> >> PACKAGE_EXTRA_ARCHS_tune-haswell =
>>> "${PACKAGE_EXTRA_ARCHS_tune-corei7} haswell"
>>> >>
>>> >>
>>> >>
>>> >
>>>
>>> --
>>> _______________________________________________
>>> yocto mailing list
>>> yocto@yoctoproject.org
>>> https://lists.yoctoproject.org/listinfo/yocto
>>>
>>
>>
>

[-- Attachment #2: Type: text/html, Size: 11342 bytes --]

Re: Cross-compiling python fails with new custom tuning

[Mark Hatle]

On 11/25/15 10:11 AM, Michael Habibi wrote:
> Well, I'm wrong yet again. A 'which python2.7' shows that it's pointing to a
> native version of Python:
> 
> *| which python2.7*
> *|
> /projects/yocto-git/build/tmp/sysroots/x86_64-linux/usr/bin/python-native/python2.7*
> |
> _PYTHON_PROJECT_BASE=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build
> _PYTHON_HOST_PLATFORM=linux2-x86_64
> PYTHONPATH=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build/build/lib.linux2-x86_64-2.7:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib/plat-linux2
> python2.7 -S -m sysconfig --generate-posix-vars ;\
> |       if test $? -ne 0 ; then \
> |               echo "generate-posix-vars failed" ; \
> |               rm -f ./pybuilddir.txt ; \
> |               exit 1 ; \
> |       fi
> | Illegal instruction (core dumped)
> | make: *** [sharedmods] Error 132
> 
> I guess one of the environmental variables being passed in is screwing up what
> it's doing. Unfortunately I don't know enough about the inner workings of Python
> to be of much help moving forward. I gave it my best shot!

The problem I tracked down was:

PYTHONPATH=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build/build/lib.linux2-x86_64-2.7

This is full of .so objects when loaded cause the illegal instruction.

I came up with a patch that I thought was going to fix it, but it's triggered
other failures.

http://lists.openembedded.org/pipermail/openembedded-core/2015-November/113341.html

Just to be clear.. this does NOT work properly in many cases.  But might give
you or someone else a clue as to how to possibly fix it.

--Mark

> 
> On Wed, Nov 25, 2015 at 10:01 AM, Michael Habibi <mikehabibi@gmail.com
> <mailto:mikehabibi@gmail.com>> wrote:
> 
>     Ah sorry, I misspoke. I walked through the Makefile and configure scripts a
>     bit more, and $(PYTHON_FOR_BUILD) should in fact be a host version of
>     Python. It seems it's not being configured correctly. Either the path is
>     setup wrong and python2.7 is not pointing to the right version, or python2.7
>     needs to be pointing to the absolute path instead of relying on $PATH.
> 
> 
> 
> 
>     On Wed, Nov 25, 2015 at 9:45 AM, Michael Habibi <mikehabibi@gmail.com
>     <mailto:mikehabibi@gmail.com>> wrote:
> 
>         For what it's worth, this is the offending portion of Makefile.pre:452:
> 
>         # Create build directory and generate the sysconfig build-time data there.
>         # pybuilddir.txt contains the name of the build dir and is used for
>         # sys.path fixup -- see Modules/getpath.c.
>         # Since this step runs before shared modules are built, try to avoid
>         bootstrap
>         # problems by creating a dummy pybuilddir.txt just to allow interpreter
>         # initialization to succeed.  It will be overwritten by generate-posix-vars
>         # or removed in case of failure.
>         pybuilddir.txt: $(BUILDPYTHON)
>         → @echo "none" > ./pybuilddir.txt
>         → $(RUNSHARED) $(PYTHON_FOR_BUILD) -S -m sysconfig --generate-posix-vars ;\
>         → if test $$? -ne 0 ; then \
>         → → echo "generate-posix-vars failed" ; \
>         → → rm -f ./pybuilddir.txt ; \
>         → → exit 1 ; \
>         → fi
> 
>         I don't know enough about the Python build to understand what it's
>         trying to do, but perhaps replacing PYTHON_FOR_BUILD with HOSTPYTHON may
>         be acceptable?
> 
>         I'm surprised this seems to work for other people, since this should be
>         failing for anyone using Python on a target platform different from
>         their host.
> 
>         On Wed, Nov 25, 2015 at 9:02 AM, Mark Hatle <mark.hatle@windriver.com
>         <mailto:mark.hatle@windriver.com>> wrote:
> 
>             On 11/24/15 3:23 PM, Mark Hatle wrote:
>             > My guess is that there is a bug in the python integration where
>             it's not
>             > realizing the host and target are different systems, so it's
>             trying to run a
>             > target program on your host.  Your host isn't haswell, so...
>             Illegal Instruction
>             > -- and the system stops.
> 
>             Just an FYI, I hit the same problem today.  I suspect it is the host
>             trying to
>             run target software and I'm looking into it.
> 
>             --Mark
> 
>             > The alternative would be something is running QEMU to execute a
>             target binary
>             > and QEMU doesn't have instruction support -- but that doesn't look
>             like the case
>             > here.
>             >
>             > --Mark
>             >
>             > On 11/24/15 3:06 PM, Michael Habibi wrote:
>             >> All,
>             >>
>             >> I added a new machine definition with tuning parameters for haswell
>             >> microarchitectures (basically just duplicated corei7 but tuned it
>             for haswell).
>             >> This seems to work correctly, but failed when running do_install
>             for python
>             >> toward the end of the build process. I am running with the Yocto
>             2.0 framework,
>             >> with very minimal changes to create a new distribution and
>             machine for our
>             >> custom embedded device. Note I had this distro configuration
>             working before, and
>             >> the only difference is I added a new machine with this tuning.
>             >>
>             >> I believe the issue is because, as part of the do_install step
>             for Python, it
>             >> attempts to run python on the local host, despite being
>             cross-compiled. However,
>             >> it is tuned for a processor that my host machine doesn't run, so
>             I get an
>             >> instruction exception.
>             >>
>             >> Did I do something weird? I figure python would be configured for
>             >> cross-compiling and therefore wouldn't try to run the target
>             version on the
>             >> host, even for sanity tests. Here is the output of the failure:
>             >>
>             >> $ tail -20
>             >>
>             /projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/temp/log.do_install.17258
>             >>
>             >> x86_64-diags-linux-ar rc libpython2.7.a Modules/threadmodule.o
>             >>  Modules/signalmodule.o  Modules/posixmodule.o  Modules/errnomodule.o
>             >>  Modules/pwdmodule.o  Modules/_sre.o  Modules/_codecsmodule.o
>             >>  Modules/_weakref.o  Modules/zipimport.o  Modules/symtablemodule.o
>             >>  Modules/md5module.o Modules/md5.o  Modules/xxsubtype.o
>             >> x86_64-diags-linux-ranlib libpython2.7.a
>             >> Modules/posixmodule.o: In function `posix_tmpnam':
>             >>
>             /projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Modules/posixmodule.c:7575:
>             >> warning: the use of `tmpnam_r' is dangerous, better use `mkstemp'
>             >> Modules/posixmodule.o: In function `posix_tempnam':
>             >>
>             /projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Modules/posixmodule.c:7522:
>             >> warning: the use of `tempnam' is dangerous, better use `mkstemp'
>             >> x86_64-diags-linux-gcc  -m64 -march=haswell -mtune=haswell
>             -mfpmath=sse -mavx2
>             >> --sysroot=/projects/yocto-git/build/tmp/sysroots/continental -Wl,-O1
>             >> -Wl,--hash-style=gnu -Wl,--as-needed -Xlinker -export-dynamic -o
>             python \
>             >>                         Modules/python.o \
>             >>                         -L. -lpython2.7 -lpthread -ldl  -lpthread
>             -lutil   -lm
>             >>
>             _PYTHON_PROJECT_BASE=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build
>             >> _PYTHON_HOST_PLATFORM=linux2-x86_64
>             >>
>             PYTHONPATH=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build/build/lib.linux2-x86_64-2.7:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib/plat-linux2
>             >> python2.7 -S -m sysconfig --generate-posix-vars ;\
>             >>         if test $? -ne 0 ; then \
>             >>                 echo "generate-posix-vars failed" ; \
>             >>                 rm -f ./pybuilddir.txt ; \
>             >>                 exit 1 ; \
>             >>         fi
>             >> Illegal instruction (core dumped)
>             >> make: *** [sharedmods] Error 132
>             >> WARNING: exit code 1 from a shell command.
>             >> ERROR: oe_runmake failed
>             >> ERROR: Function failed: do_install (log file is located at
>             >>
>             /projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/temp/log.do_install.17258)
>             >>
>             >> Here is my tune-haswell.inc (ignore some copy/paste comment
>             issues right now =):
>             >>
>             >> # Settings for the GCC(1) cpu-type "haswell":
>             >> #
>             >> #     Intel Core i7 CPU with 64-bit extensions, MOVBE, MMX, SSE,
>             SSE2, SSE3,·
>             >> #     SSSE3, SSE4.1, SSE4.2, POPCNT, AVX, AVX2, AES, PCLMUL,
>             FSGSBASE,
>             >> #     RDRND, FMA, BMI, BMI2 and F16C instruction set support.
>             >> #
>             >> # This tune is recommended for Intel Nehalem and Silvermont (e.g.
>             Bay Trail) CPUs
>             >> # (and beyond).
>             >> #
>             >> DEFAULTTUNE ?= "haswell"
>             >>
>             >> # Pull in the previous tune in to pull in PACKAGE_EXTRA_ARCHS
>             >> require conf/machine/include/tune-corei7.inc
>             >>
>             >> # Extra tune features
>             >> TUNEVALID[haswell] = "Enable haswell specific processor
>             optimizations"
>             >> TUNE_CCARGS .= "${@bb.utils.contains("TUNE_FEATURES", "haswell", "
>             >> -march=haswell -mtune=haswell -mfpmath=sse -mavx2", "", d)}"
>             >>
>             >> # Extra tune selections
>             >> AVAILTUNES += "haswell"
>             >> TUNE_FEATURES_tune-haswell = "${TUNE_FEATURES_tune-x86-64} haswell"
>             >> BASE_LIB_tune-haswell = "lib"
>             >> TUNE_PKGARCH_tune-haswell = "haswell"
>             >> PACKAGE_EXTRA_ARCHS_tune-haswell =
>             "${PACKAGE_EXTRA_ARCHS_tune-corei7} haswell"
>             >>
>             >>
>             >>
>             >
> 
>             --
>             _______________________________________________
>             yocto mailing list
>             yocto@yoctoproject.org <mailto:yocto@yoctoproject.org>
>             https://lists.yoctoproject.org/listinfo/yocto
> 
> 
> 
> 

Re: Cross-compiling python fails with new custom tuning

[Michael Habibi]

[-- Attachment #1: Type: text/plain, Size: 13231 bytes --]

Mark,

I ran the same command, including the offending PYTHONPATH, from the shell
without error. Any reason why that would work, despite it not working from
within the Makefile? I was thinking it was another environment variable
that was causing the issue (one that's not set in my bash environment by
default). My $PATH didn't have python-native so I called it using an
absolute path, but otherwise it's the same command run from the Makefile
(parsed, of course):

bash$
_PYTHON_PROJECT_BASE=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build
_PYTHON_HOST_PLATFORM=linux2-x86_64
PYTHONPATH=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build/build/lib.linux2-x86_64-2.7:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib/plat-linux2
/projects/yocto-git/build/tmp/sysroots/x86_64-linux/usr/bin/python-native/python2.7
-S -m sysconfig --generate-posix-vars
bash$ ls pybuilddir.txt
pybuilddir.txt

Maybe I'm missing something.


On Wed, Nov 25, 2015 at 4:38 PM, Mark Hatle <mark.hatle@windriver.com>
wrote:

> On 11/25/15 10:11 AM, Michael Habibi wrote:
> > Well, I'm wrong yet again. A 'which python2.7' shows that it's pointing
> to a
> > native version of Python:
> >
> > *| which python2.7*
> > *|
> >
> /projects/yocto-git/build/tmp/sysroots/x86_64-linux/usr/bin/python-native/python2.7*
> > |
> >
> _PYTHON_PROJECT_BASE=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build
> > _PYTHON_HOST_PLATFORM=linux2-x86_64
> >
> PYTHONPATH=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build/build/lib.linux2-x86_64-2.7:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib/plat-linux2
> > python2.7 -S -m sysconfig --generate-posix-vars ;\
> > |       if test $? -ne 0 ; then \
> > |               echo "generate-posix-vars failed" ; \
> > |               rm -f ./pybuilddir.txt ; \
> > |               exit 1 ; \
> > |       fi
> > | Illegal instruction (core dumped)
> > | make: *** [sharedmods] Error 132
> >
> > I guess one of the environmental variables being passed in is screwing
> up what
> > it's doing. Unfortunately I don't know enough about the inner workings
> of Python
> > to be of much help moving forward. I gave it my best shot!
>
> The problem I tracked down was:
>
>
> PYTHONPATH=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build/build/lib.linux2-x86_64-2.7
>
> This is full of .so objects when loaded cause the illegal instruction.
>
> I came up with a patch that I thought was going to fix it, but it's
> triggered
> other failures.
>
>
> http://lists.openembedded.org/pipermail/openembedded-core/2015-November/113341.html
>
> Just to be clear.. this does NOT work properly in many cases.  But might
> give
> you or someone else a clue as to how to possibly fix it.
>
> --Mark
>
> >
> > On Wed, Nov 25, 2015 at 10:01 AM, Michael Habibi <mikehabibi@gmail.com
> > <mailto:mikehabibi@gmail.com>> wrote:
> >
> >     Ah sorry, I misspoke. I walked through the Makefile and configure
> scripts a
> >     bit more, and $(PYTHON_FOR_BUILD) should in fact be a host version of
> >     Python. It seems it's not being configured correctly. Either the
> path is
> >     setup wrong and python2.7 is not pointing to the right version, or
> python2.7
> >     needs to be pointing to the absolute path instead of relying on
> $PATH.
> >
> >
> >
> >
> >     On Wed, Nov 25, 2015 at 9:45 AM, Michael Habibi <
> mikehabibi@gmail.com
> >     <mailto:mikehabibi@gmail.com>> wrote:
> >
> >         For what it's worth, this is the offending portion of
> Makefile.pre:452:
> >
> >         # Create build directory and generate the sysconfig build-time
> data there.
> >         # pybuilddir.txt contains the name of the build dir and is used
> for
> >         # sys.path fixup -- see Modules/getpath.c.
> >         # Since this step runs before shared modules are built, try to
> avoid
> >         bootstrap
> >         # problems by creating a dummy pybuilddir.txt just to allow
> interpreter
> >         # initialization to succeed.  It will be overwritten by
> generate-posix-vars
> >         # or removed in case of failure.
> >         pybuilddir.txt: $(BUILDPYTHON)
> >         → @echo "none" > ./pybuilddir.txt
> >         → $(RUNSHARED) $(PYTHON_FOR_BUILD) -S -m sysconfig
> --generate-posix-vars ;\
> >         → if test $$? -ne 0 ; then \
> >         → → echo "generate-posix-vars failed" ; \
> >         → → rm -f ./pybuilddir.txt ; \
> >         → → exit 1 ; \
> >         → fi
> >
> >         I don't know enough about the Python build to understand what
> it's
> >         trying to do, but perhaps replacing PYTHON_FOR_BUILD with
> HOSTPYTHON may
> >         be acceptable?
> >
> >         I'm surprised this seems to work for other people, since this
> should be
> >         failing for anyone using Python on a target platform different
> from
> >         their host.
> >
> >         On Wed, Nov 25, 2015 at 9:02 AM, Mark Hatle <
> mark.hatle@windriver.com
> >         <mailto:mark.hatle@windriver.com>> wrote:
> >
> >             On 11/24/15 3:23 PM, Mark Hatle wrote:
> >             > My guess is that there is a bug in the python integration
> where
> >             it's not
> >             > realizing the host and target are different systems, so
> it's
> >             trying to run a
> >             > target program on your host.  Your host isn't haswell,
> so...
> >             Illegal Instruction
> >             > -- and the system stops.
> >
> >             Just an FYI, I hit the same problem today.  I suspect it is
> the host
> >             trying to
> >             run target software and I'm looking into it.
> >
> >             --Mark
> >
> >             > The alternative would be something is running QEMU to
> execute a
> >             target binary
> >             > and QEMU doesn't have instruction support -- but that
> doesn't look
> >             like the case
> >             > here.
> >             >
> >             > --Mark
> >             >
> >             > On 11/24/15 3:06 PM, Michael Habibi wrote:
> >             >> All,
> >             >>
> >             >> I added a new machine definition with tuning parameters
> for haswell
> >             >> microarchitectures (basically just duplicated corei7 but
> tuned it
> >             for haswell).
> >             >> This seems to work correctly, but failed when running
> do_install
> >             for python
> >             >> toward the end of the build process. I am running with
> the Yocto
> >             2.0 framework,
> >             >> with very minimal changes to create a new distribution and
> >             machine for our
> >             >> custom embedded device. Note I had this distro
> configuration
> >             working before, and
> >             >> the only difference is I added a new machine with this
> tuning.
> >             >>
> >             >> I believe the issue is because, as part of the do_install
> step
> >             for Python, it
> >             >> attempts to run python on the local host, despite being
> >             cross-compiled. However,
> >             >> it is tuned for a processor that my host machine doesn't
> run, so
> >             I get an
> >             >> instruction exception.
> >             >>
> >             >> Did I do something weird? I figure python would be
> configured for
> >             >> cross-compiling and therefore wouldn't try to run the
> target
> >             version on the
> >             >> host, even for sanity tests. Here is the output of the
> failure:
> >             >>
> >             >> $ tail -20
> >             >>
> >
>  /projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/temp/log.do_install.17258
> >             >>
> >             >> x86_64-diags-linux-ar rc libpython2.7.a
> Modules/threadmodule.o
> >             >>  Modules/signalmodule.o  Modules/posixmodule.o
> Modules/errnomodule.o
> >             >>  Modules/pwdmodule.o  Modules/_sre.o
> Modules/_codecsmodule.o
> >             >>  Modules/_weakref.o  Modules/zipimport.o
> Modules/symtablemodule.o
> >             >>  Modules/md5module.o Modules/md5.o  Modules/xxsubtype.o
> >             >> x86_64-diags-linux-ranlib libpython2.7.a
> >             >> Modules/posixmodule.o: In function `posix_tmpnam':
> >             >>
> >
>  /projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Modules/posixmodule.c:7575:
> >             >> warning: the use of `tmpnam_r' is dangerous, better use
> `mkstemp'
> >             >> Modules/posixmodule.o: In function `posix_tempnam':
> >             >>
> >
>  /projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Modules/posixmodule.c:7522:
> >             >> warning: the use of `tempnam' is dangerous, better use
> `mkstemp'
> >             >> x86_64-diags-linux-gcc  -m64 -march=haswell -mtune=haswell
> >             -mfpmath=sse -mavx2
> >             >>
> --sysroot=/projects/yocto-git/build/tmp/sysroots/continental -Wl,-O1
> >             >> -Wl,--hash-style=gnu -Wl,--as-needed -Xlinker
> -export-dynamic -o
> >             python \
> >             >>                         Modules/python.o \
> >             >>                         -L. -lpython2.7 -lpthread -ldl
> -lpthread
> >             -lutil   -lm
> >             >>
> >
>  _PYTHON_PROJECT_BASE=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build
> >             >> _PYTHON_HOST_PLATFORM=linux2-x86_64
> >             >>
> >
>  PYTHONPATH=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build/build/lib.linux2-x86_64-2.7:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib/plat-linux2
> >             >> python2.7 -S -m sysconfig --generate-posix-vars ;\
> >             >>         if test $? -ne 0 ; then \
> >             >>                 echo "generate-posix-vars failed" ; \
> >             >>                 rm -f ./pybuilddir.txt ; \
> >             >>                 exit 1 ; \
> >             >>         fi
> >             >> Illegal instruction (core dumped)
> >             >> make: *** [sharedmods] Error 132
> >             >> WARNING: exit code 1 from a shell command.
> >             >> ERROR: oe_runmake failed
> >             >> ERROR: Function failed: do_install (log file is located at
> >             >>
> >
>  /projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/temp/log.do_install.17258)
> >             >>
> >             >> Here is my tune-haswell.inc (ignore some copy/paste
> comment
> >             issues right now =):
> >             >>
> >             >> # Settings for the GCC(1) cpu-type "haswell":
> >             >> #
> >             >> #     Intel Core i7 CPU with 64-bit extensions, MOVBE,
> MMX, SSE,
> >             SSE2, SSE3,·
> >             >> #     SSSE3, SSE4.1, SSE4.2, POPCNT, AVX, AVX2, AES,
> PCLMUL,
> >             FSGSBASE,
> >             >> #     RDRND, FMA, BMI, BMI2 and F16C instruction set
> support.
> >             >> #
> >             >> # This tune is recommended for Intel Nehalem and
> Silvermont (e.g.
> >             Bay Trail) CPUs
> >             >> # (and beyond).
> >             >> #
> >             >> DEFAULTTUNE ?= "haswell"
> >             >>
> >             >> # Pull in the previous tune in to pull in
> PACKAGE_EXTRA_ARCHS
> >             >> require conf/machine/include/tune-corei7.inc
> >             >>
> >             >> # Extra tune features
> >             >> TUNEVALID[haswell] = "Enable haswell specific processor
> >             optimizations"
> >             >> TUNE_CCARGS .= "${@bb.utils.contains("TUNE_FEATURES",
> "haswell", "
> >             >> -march=haswell -mtune=haswell -mfpmath=sse -mavx2", "",
> d)}"
> >             >>
> >             >> # Extra tune selections
> >             >> AVAILTUNES += "haswell"
> >             >> TUNE_FEATURES_tune-haswell =
> "${TUNE_FEATURES_tune-x86-64} haswell"
> >             >> BASE_LIB_tune-haswell = "lib"
> >             >> TUNE_PKGARCH_tune-haswell = "haswell"
> >             >> PACKAGE_EXTRA_ARCHS_tune-haswell =
> >             "${PACKAGE_EXTRA_ARCHS_tune-corei7} haswell"
> >             >>
> >             >>
> >             >>
> >             >
> >
> >             --
> >             _______________________________________________
> >             yocto mailing list
> >             yocto@yoctoproject.org <mailto:yocto@yoctoproject.org>
> >             https://lists.yoctoproject.org/listinfo/yocto
> >
> >
> >
> >
>
>

[-- Attachment #2: Type: text/html, Size: 17418 bytes --]

Re: Cross-compiling python fails with new custom tuning

[Michael Habibi]

[-- Attachment #1: Type: text/plain, Size: 14864 bytes --]

Mark, I am continuing to look at this. I hope you don't mind if I keep
updating this thread with my investigation.

First, I figured out my confusion. I was associating the wrong Makefile
command with the illegal instruction. It was actually the command under the
recipe for 'sharedmods', but it was silenced in the Makefile so I assumed
it was another line. You're right that the path with the *.so's is the
offending path. I've scoured online patches and can't understand how this
mechanism works outside of Yocto. I figure people trying to compile for ARM
would have similar issues. There seems to be a fundamental problem with how
Python sets up PYTHONPATH using the pybuilddir file and $abs_builddir.
However, other people seem to have successfully built python for other
architectures, so I'm still confused.

I hardcoded a fix for PYTHONPATH to use the host lib-dynload directory, and
the python recipe successfully passed. Can you elaborate on the issues you
were having with your patch? The only difference is that I maintained the
other two directories in $PYTHONPATH,
e.g. $(srcdir)/Lib:$(srcdir)/Lib/plat-$(MACHDEP), and only replaced the
offending one.

On Wed, Nov 25, 2015 at 4:58 PM, Michael Habibi <mikehabibi@gmail.com>
wrote:

> Mark,
>
> I ran the same command, including the offending PYTHONPATH, from the shell
> without error. Any reason why that would work, despite it not working from
> within the Makefile? I was thinking it was another environment variable
> that was causing the issue (one that's not set in my bash environment by
> default). My $PATH didn't have python-native so I called it using an
> absolute path, but otherwise it's the same command run from the Makefile
> (parsed, of course):
>
> bash$
> _PYTHON_PROJECT_BASE=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build
> _PYTHON_HOST_PLATFORM=linux2-x86_64
> PYTHONPATH=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build/build/lib.linux2-x86_64-2.7:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib/plat-linux2
> /projects/yocto-git/build/tmp/sysroots/x86_64-linux/usr/bin/python-native/python2.7
> -S -m sysconfig --generate-posix-vars
> bash$ ls pybuilddir.txt
> pybuilddir.txt
>
> Maybe I'm missing something.
>
>
> On Wed, Nov 25, 2015 at 4:38 PM, Mark Hatle <mark.hatle@windriver.com>
> wrote:
>
>> On 11/25/15 10:11 AM, Michael Habibi wrote:
>> > Well, I'm wrong yet again. A 'which python2.7' shows that it's pointing
>> to a
>> > native version of Python:
>> >
>> > *| which python2.7*
>> > *|
>> >
>> /projects/yocto-git/build/tmp/sysroots/x86_64-linux/usr/bin/python-native/python2.7*
>> > |
>> >
>> _PYTHON_PROJECT_BASE=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build
>> > _PYTHON_HOST_PLATFORM=linux2-x86_64
>> >
>> PYTHONPATH=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build/build/lib.linux2-x86_64-2.7:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib/plat-linux2
>> > python2.7 -S -m sysconfig --generate-posix-vars ;\
>> > |       if test $? -ne 0 ; then \
>> > |               echo "generate-posix-vars failed" ; \
>> > |               rm -f ./pybuilddir.txt ; \
>> > |               exit 1 ; \
>> > |       fi
>> > | Illegal instruction (core dumped)
>> > | make: *** [sharedmods] Error 132
>> >
>> > I guess one of the environmental variables being passed in is screwing
>> up what
>> > it's doing. Unfortunately I don't know enough about the inner workings
>> of Python
>> > to be of much help moving forward. I gave it my best shot!
>>
>> The problem I tracked down was:
>>
>>
>> PYTHONPATH=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build/build/lib.linux2-x86_64-2.7
>>
>> This is full of .so objects when loaded cause the illegal instruction.
>>
>> I came up with a patch that I thought was going to fix it, but it's
>> triggered
>> other failures.
>>
>>
>> http://lists.openembedded.org/pipermail/openembedded-core/2015-November/113341.html
>>
>> Just to be clear.. this does NOT work properly in many cases.  But might
>> give
>> you or someone else a clue as to how to possibly fix it.
>>
>> --Mark
>>
>> >
>> > On Wed, Nov 25, 2015 at 10:01 AM, Michael Habibi <mikehabibi@gmail.com
>> > <mailto:mikehabibi@gmail.com>> wrote:
>> >
>> >     Ah sorry, I misspoke. I walked through the Makefile and configure
>> scripts a
>> >     bit more, and $(PYTHON_FOR_BUILD) should in fact be a host version
>> of
>> >     Python. It seems it's not being configured correctly. Either the
>> path is
>> >     setup wrong and python2.7 is not pointing to the right version, or
>> python2.7
>> >     needs to be pointing to the absolute path instead of relying on
>> $PATH.
>> >
>> >
>> >
>> >
>> >     On Wed, Nov 25, 2015 at 9:45 AM, Michael Habibi <
>> mikehabibi@gmail.com
>> >     <mailto:mikehabibi@gmail.com>> wrote:
>> >
>> >         For what it's worth, this is the offending portion of
>> Makefile.pre:452:
>> >
>> >         # Create build directory and generate the sysconfig build-time
>> data there.
>> >         # pybuilddir.txt contains the name of the build dir and is used
>> for
>> >         # sys.path fixup -- see Modules/getpath.c.
>> >         # Since this step runs before shared modules are built, try to
>> avoid
>> >         bootstrap
>> >         # problems by creating a dummy pybuilddir.txt just to allow
>> interpreter
>> >         # initialization to succeed.  It will be overwritten by
>> generate-posix-vars
>> >         # or removed in case of failure.
>> >         pybuilddir.txt: $(BUILDPYTHON)
>> >         → @echo "none" > ./pybuilddir.txt
>> >         → $(RUNSHARED) $(PYTHON_FOR_BUILD) -S -m sysconfig
>> --generate-posix-vars ;\
>> >         → if test $$? -ne 0 ; then \
>> >         → → echo "generate-posix-vars failed" ; \
>> >         → → rm -f ./pybuilddir.txt ; \
>> >         → → exit 1 ; \
>> >         → fi
>> >
>> >         I don't know enough about the Python build to understand what
>> it's
>> >         trying to do, but perhaps replacing PYTHON_FOR_BUILD with
>> HOSTPYTHON may
>> >         be acceptable?
>> >
>> >         I'm surprised this seems to work for other people, since this
>> should be
>> >         failing for anyone using Python on a target platform different
>> from
>> >         their host.
>> >
>> >         On Wed, Nov 25, 2015 at 9:02 AM, Mark Hatle <
>> mark.hatle@windriver.com
>> >         <mailto:mark.hatle@windriver.com>> wrote:
>> >
>> >             On 11/24/15 3:23 PM, Mark Hatle wrote:
>> >             > My guess is that there is a bug in the python integration
>> where
>> >             it's not
>> >             > realizing the host and target are different systems, so
>> it's
>> >             trying to run a
>> >             > target program on your host.  Your host isn't haswell,
>> so...
>> >             Illegal Instruction
>> >             > -- and the system stops.
>> >
>> >             Just an FYI, I hit the same problem today.  I suspect it is
>> the host
>> >             trying to
>> >             run target software and I'm looking into it.
>> >
>> >             --Mark
>> >
>> >             > The alternative would be something is running QEMU to
>> execute a
>> >             target binary
>> >             > and QEMU doesn't have instruction support -- but that
>> doesn't look
>> >             like the case
>> >             > here.
>> >             >
>> >             > --Mark
>> >             >
>> >             > On 11/24/15 3:06 PM, Michael Habibi wrote:
>> >             >> All,
>> >             >>
>> >             >> I added a new machine definition with tuning parameters
>> for haswell
>> >             >> microarchitectures (basically just duplicated corei7 but
>> tuned it
>> >             for haswell).
>> >             >> This seems to work correctly, but failed when running
>> do_install
>> >             for python
>> >             >> toward the end of the build process. I am running with
>> the Yocto
>> >             2.0 framework,
>> >             >> with very minimal changes to create a new distribution
>> and
>> >             machine for our
>> >             >> custom embedded device. Note I had this distro
>> configuration
>> >             working before, and
>> >             >> the only difference is I added a new machine with this
>> tuning.
>> >             >>
>> >             >> I believe the issue is because, as part of the
>> do_install step
>> >             for Python, it
>> >             >> attempts to run python on the local host, despite being
>> >             cross-compiled. However,
>> >             >> it is tuned for a processor that my host machine doesn't
>> run, so
>> >             I get an
>> >             >> instruction exception.
>> >             >>
>> >             >> Did I do something weird? I figure python would be
>> configured for
>> >             >> cross-compiling and therefore wouldn't try to run the
>> target
>> >             version on the
>> >             >> host, even for sanity tests. Here is the output of the
>> failure:
>> >             >>
>> >             >> $ tail -20
>> >             >>
>> >
>>  /projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/temp/log.do_install.17258
>> >             >>
>> >             >> x86_64-diags-linux-ar rc libpython2.7.a
>> Modules/threadmodule.o
>> >             >>  Modules/signalmodule.o  Modules/posixmodule.o
>> Modules/errnomodule.o
>> >             >>  Modules/pwdmodule.o  Modules/_sre.o
>> Modules/_codecsmodule.o
>> >             >>  Modules/_weakref.o  Modules/zipimport.o
>> Modules/symtablemodule.o
>> >             >>  Modules/md5module.o Modules/md5.o  Modules/xxsubtype.o
>> >             >> x86_64-diags-linux-ranlib libpython2.7.a
>> >             >> Modules/posixmodule.o: In function `posix_tmpnam':
>> >             >>
>> >
>>  /projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Modules/posixmodule.c:7575:
>> >             >> warning: the use of `tmpnam_r' is dangerous, better use
>> `mkstemp'
>> >             >> Modules/posixmodule.o: In function `posix_tempnam':
>> >             >>
>> >
>>  /projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Modules/posixmodule.c:7522:
>> >             >> warning: the use of `tempnam' is dangerous, better use
>> `mkstemp'
>> >             >> x86_64-diags-linux-gcc  -m64 -march=haswell
>> -mtune=haswell
>> >             -mfpmath=sse -mavx2
>> >             >>
>> --sysroot=/projects/yocto-git/build/tmp/sysroots/continental -Wl,-O1
>> >             >> -Wl,--hash-style=gnu -Wl,--as-needed -Xlinker
>> -export-dynamic -o
>> >             python \
>> >             >>                         Modules/python.o \
>> >             >>                         -L. -lpython2.7 -lpthread -ldl
>> -lpthread
>> >             -lutil   -lm
>> >             >>
>> >
>>  _PYTHON_PROJECT_BASE=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build
>> >             >> _PYTHON_HOST_PLATFORM=linux2-x86_64
>> >             >>
>> >
>>  PYTHONPATH=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build/build/lib.linux2-x86_64-2.7:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib/plat-linux2
>> >             >> python2.7 -S -m sysconfig --generate-posix-vars ;\
>> >             >>         if test $? -ne 0 ; then \
>> >             >>                 echo "generate-posix-vars failed" ; \
>> >             >>                 rm -f ./pybuilddir.txt ; \
>> >             >>                 exit 1 ; \
>> >             >>         fi
>> >             >> Illegal instruction (core dumped)
>> >             >> make: *** [sharedmods] Error 132
>> >             >> WARNING: exit code 1 from a shell command.
>> >             >> ERROR: oe_runmake failed
>> >             >> ERROR: Function failed: do_install (log file is located
>> at
>> >             >>
>> >
>>  /projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/temp/log.do_install.17258)
>> >             >>
>> >             >> Here is my tune-haswell.inc (ignore some copy/paste
>> comment
>> >             issues right now =):
>> >             >>
>> >             >> # Settings for the GCC(1) cpu-type "haswell":
>> >             >> #
>> >             >> #     Intel Core i7 CPU with 64-bit extensions, MOVBE,
>> MMX, SSE,
>> >             SSE2, SSE3,·
>> >             >> #     SSSE3, SSE4.1, SSE4.2, POPCNT, AVX, AVX2, AES,
>> PCLMUL,
>> >             FSGSBASE,
>> >             >> #     RDRND, FMA, BMI, BMI2 and F16C instruction set
>> support.
>> >             >> #
>> >             >> # This tune is recommended for Intel Nehalem and
>> Silvermont (e.g.
>> >             Bay Trail) CPUs
>> >             >> # (and beyond).
>> >             >> #
>> >             >> DEFAULTTUNE ?= "haswell"
>> >             >>
>> >             >> # Pull in the previous tune in to pull in
>> PACKAGE_EXTRA_ARCHS
>> >             >> require conf/machine/include/tune-corei7.inc
>> >             >>
>> >             >> # Extra tune features
>> >             >> TUNEVALID[haswell] = "Enable haswell specific processor
>> >             optimizations"
>> >             >> TUNE_CCARGS .= "${@bb.utils.contains("TUNE_FEATURES",
>> "haswell", "
>> >             >> -march=haswell -mtune=haswell -mfpmath=sse -mavx2", "",
>> d)}"
>> >             >>
>> >             >> # Extra tune selections
>> >             >> AVAILTUNES += "haswell"
>> >             >> TUNE_FEATURES_tune-haswell =
>> "${TUNE_FEATURES_tune-x86-64} haswell"
>> >             >> BASE_LIB_tune-haswell = "lib"
>> >             >> TUNE_PKGARCH_tune-haswell = "haswell"
>> >             >> PACKAGE_EXTRA_ARCHS_tune-haswell =
>> >             "${PACKAGE_EXTRA_ARCHS_tune-corei7} haswell"
>> >             >>
>> >             >>
>> >             >>
>> >             >
>> >
>> >             --
>> >             _______________________________________________
>> >             yocto mailing list
>> >             yocto@yoctoproject.org <mailto:yocto@yoctoproject.org>
>> >             https://lists.yoctoproject.org/listinfo/yocto
>> >
>> >
>> >
>> >
>>
>>
>

[-- Attachment #2: Type: text/html, Size: 19158 bytes --]

Re: Cross-compiling python fails with new custom tuning

[Michael Habibi]

[-- Attachment #1: Type: text/plain, Size: 17296 bytes --]

OK I've dug into this further today, and here are some findings:

With my patch (which is similar to yours), it fails do_compile/package_qa
because of poisoned system directories during the compilation steps. This
approach seems to be a bust. Originally when I tested it, I hacked up the
Makefile and only re-ran do_install onward, and didn't re-run do_compile
(because I had to create the patch). Once I created the patch and re-ran
from clean, I got some errors.

It appears the only reason it is using that directory in PYTHONPATH at all
(the one generated from the shell command and pybuilddir.txt) is for a
potential sysconfig file in there. I tracked it down to a comment in a bug
on Python's bug tracker:

http://bugs.python.org/issue15484#msg180577

If we change that directory, it will pull in the sysconfig settings from
the native Python configure (probably defaults to native sysconfig settings
because the PATH doesn't include the overrides), and then do some really
bad things. The question then became: in the unmodified build, why does
do_install fail but do_compile doesn't? They both seem to be using
sharedmods recipe which uses $PYTHONPATH. The answer is that when we run
do_compile the first time around, the offending *.so's are not generated
yet, and thus are not included by python-native. The second time around (in
do_install), the *.so's are in the path and so python-native will use them
and override its own.

My latest experiment is to remove that location altogether from PYTHONPATH
and see if it works (neither the shell-generated directory in python-cross,
nor the lib-dynload directory from python-native). The compile works the
first time through despite there being no sysconfig data, so I'm wondering
if the install step will also work without finding any data. python-native
should continue to work with its internal settings to find modules and
libraries.

To be continued tomorrow, I'm sure...

On Mon, Nov 30, 2015 at 11:37 AM, Michael Habibi <mikehabibi@gmail.com>
wrote:

> Mark, I am continuing to look at this. I hope you don't mind if I keep
> updating this thread with my investigation.
>
> First, I figured out my confusion. I was associating the wrong Makefile
> command with the illegal instruction. It was actually the command under the
> recipe for 'sharedmods', but it was silenced in the Makefile so I assumed
> it was another line. You're right that the path with the *.so's is the
> offending path. I've scoured online patches and can't understand how this
> mechanism works outside of Yocto. I figure people trying to compile for ARM
> would have similar issues. There seems to be a fundamental problem with how
> Python sets up PYTHONPATH using the pybuilddir file and $abs_builddir.
> However, other people seem to have successfully built python for other
> architectures, so I'm still confused.
>
> I hardcoded a fix for PYTHONPATH to use the host lib-dynload directory,
> and the python recipe successfully passed. Can you elaborate on the issues
> you were having with your patch? The only difference is that I maintained
> the other two directories in $PYTHONPATH,
> e.g. $(srcdir)/Lib:$(srcdir)/Lib/plat-$(MACHDEP), and only replaced the
> offending one.
>
> On Wed, Nov 25, 2015 at 4:58 PM, Michael Habibi <mikehabibi@gmail.com>
> wrote:
>
>> Mark,
>>
>> I ran the same command, including the offending PYTHONPATH, from the
>> shell without error. Any reason why that would work, despite it not working
>> from within the Makefile? I was thinking it was another environment
>> variable that was causing the issue (one that's not set in my bash
>> environment by default). My $PATH didn't have python-native so I called it
>> using an absolute path, but otherwise it's the same command run from the
>> Makefile (parsed, of course):
>>
>> bash$
>> _PYTHON_PROJECT_BASE=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build
>> _PYTHON_HOST_PLATFORM=linux2-x86_64
>> PYTHONPATH=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build/build/lib.linux2-x86_64-2.7:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib/plat-linux2
>> /projects/yocto-git/build/tmp/sysroots/x86_64-linux/usr/bin/python-native/python2.7
>> -S -m sysconfig --generate-posix-vars
>> bash$ ls pybuilddir.txt
>> pybuilddir.txt
>>
>> Maybe I'm missing something.
>>
>>
>> On Wed, Nov 25, 2015 at 4:38 PM, Mark Hatle <mark.hatle@windriver.com>
>> wrote:
>>
>>> On 11/25/15 10:11 AM, Michael Habibi wrote:
>>> > Well, I'm wrong yet again. A 'which python2.7' shows that it's
>>> pointing to a
>>> > native version of Python:
>>> >
>>> > *| which python2.7*
>>> > *|
>>> >
>>> /projects/yocto-git/build/tmp/sysroots/x86_64-linux/usr/bin/python-native/python2.7*
>>> > |
>>> >
>>> _PYTHON_PROJECT_BASE=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build
>>> > _PYTHON_HOST_PLATFORM=linux2-x86_64
>>> >
>>> PYTHONPATH=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build/build/lib.linux2-x86_64-2.7:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib/plat-linux2
>>> > python2.7 -S -m sysconfig --generate-posix-vars ;\
>>> > |       if test $? -ne 0 ; then \
>>> > |               echo "generate-posix-vars failed" ; \
>>> > |               rm -f ./pybuilddir.txt ; \
>>> > |               exit 1 ; \
>>> > |       fi
>>> > | Illegal instruction (core dumped)
>>> > | make: *** [sharedmods] Error 132
>>> >
>>> > I guess one of the environmental variables being passed in is screwing
>>> up what
>>> > it's doing. Unfortunately I don't know enough about the inner workings
>>> of Python
>>> > to be of much help moving forward. I gave it my best shot!
>>>
>>> The problem I tracked down was:
>>>
>>>
>>> PYTHONPATH=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build/build/lib.linux2-x86_64-2.7
>>>
>>> This is full of .so objects when loaded cause the illegal instruction.
>>>
>>> I came up with a patch that I thought was going to fix it, but it's
>>> triggered
>>> other failures.
>>>
>>>
>>> http://lists.openembedded.org/pipermail/openembedded-core/2015-November/113341.html
>>>
>>> Just to be clear.. this does NOT work properly in many cases.  But might
>>> give
>>> you or someone else a clue as to how to possibly fix it.
>>>
>>> --Mark
>>>
>>> >
>>> > On Wed, Nov 25, 2015 at 10:01 AM, Michael Habibi <mikehabibi@gmail.com
>>> > <mailto:mikehabibi@gmail.com>> wrote:
>>> >
>>> >     Ah sorry, I misspoke. I walked through the Makefile and configure
>>> scripts a
>>> >     bit more, and $(PYTHON_FOR_BUILD) should in fact be a host version
>>> of
>>> >     Python. It seems it's not being configured correctly. Either the
>>> path is
>>> >     setup wrong and python2.7 is not pointing to the right version, or
>>> python2.7
>>> >     needs to be pointing to the absolute path instead of relying on
>>> $PATH.
>>> >
>>> >
>>> >
>>> >
>>> >     On Wed, Nov 25, 2015 at 9:45 AM, Michael Habibi <
>>> mikehabibi@gmail.com
>>> >     <mailto:mikehabibi@gmail.com>> wrote:
>>> >
>>> >         For what it's worth, this is the offending portion of
>>> Makefile.pre:452:
>>> >
>>> >         # Create build directory and generate the sysconfig build-time
>>> data there.
>>> >         # pybuilddir.txt contains the name of the build dir and is
>>> used for
>>> >         # sys.path fixup -- see Modules/getpath.c.
>>> >         # Since this step runs before shared modules are built, try to
>>> avoid
>>> >         bootstrap
>>> >         # problems by creating a dummy pybuilddir.txt just to allow
>>> interpreter
>>> >         # initialization to succeed.  It will be overwritten by
>>> generate-posix-vars
>>> >         # or removed in case of failure.
>>> >         pybuilddir.txt: $(BUILDPYTHON)
>>> >         → @echo "none" > ./pybuilddir.txt
>>> >         → $(RUNSHARED) $(PYTHON_FOR_BUILD) -S -m sysconfig
>>> --generate-posix-vars ;\
>>> >         → if test $$? -ne 0 ; then \
>>> >         → → echo "generate-posix-vars failed" ; \
>>> >         → → rm -f ./pybuilddir.txt ; \
>>> >         → → exit 1 ; \
>>> >         → fi
>>> >
>>> >         I don't know enough about the Python build to understand what
>>> it's
>>> >         trying to do, but perhaps replacing PYTHON_FOR_BUILD with
>>> HOSTPYTHON may
>>> >         be acceptable?
>>> >
>>> >         I'm surprised this seems to work for other people, since this
>>> should be
>>> >         failing for anyone using Python on a target platform different
>>> from
>>> >         their host.
>>> >
>>> >         On Wed, Nov 25, 2015 at 9:02 AM, Mark Hatle <
>>> mark.hatle@windriver.com
>>> >         <mailto:mark.hatle@windriver.com>> wrote:
>>> >
>>> >             On 11/24/15 3:23 PM, Mark Hatle wrote:
>>> >             > My guess is that there is a bug in the python
>>> integration where
>>> >             it's not
>>> >             > realizing the host and target are different systems, so
>>> it's
>>> >             trying to run a
>>> >             > target program on your host.  Your host isn't haswell,
>>> so...
>>> >             Illegal Instruction
>>> >             > -- and the system stops.
>>> >
>>> >             Just an FYI, I hit the same problem today.  I suspect it
>>> is the host
>>> >             trying to
>>> >             run target software and I'm looking into it.
>>> >
>>> >             --Mark
>>> >
>>> >             > The alternative would be something is running QEMU to
>>> execute a
>>> >             target binary
>>> >             > and QEMU doesn't have instruction support -- but that
>>> doesn't look
>>> >             like the case
>>> >             > here.
>>> >             >
>>> >             > --Mark
>>> >             >
>>> >             > On 11/24/15 3:06 PM, Michael Habibi wrote:
>>> >             >> All,
>>> >             >>
>>> >             >> I added a new machine definition with tuning parameters
>>> for haswell
>>> >             >> microarchitectures (basically just duplicated corei7
>>> but tuned it
>>> >             for haswell).
>>> >             >> This seems to work correctly, but failed when running
>>> do_install
>>> >             for python
>>> >             >> toward the end of the build process. I am running with
>>> the Yocto
>>> >             2.0 framework,
>>> >             >> with very minimal changes to create a new distribution
>>> and
>>> >             machine for our
>>> >             >> custom embedded device. Note I had this distro
>>> configuration
>>> >             working before, and
>>> >             >> the only difference is I added a new machine with this
>>> tuning.
>>> >             >>
>>> >             >> I believe the issue is because, as part of the
>>> do_install step
>>> >             for Python, it
>>> >             >> attempts to run python on the local host, despite being
>>> >             cross-compiled. However,
>>> >             >> it is tuned for a processor that my host machine
>>> doesn't run, so
>>> >             I get an
>>> >             >> instruction exception.
>>> >             >>
>>> >             >> Did I do something weird? I figure python would be
>>> configured for
>>> >             >> cross-compiling and therefore wouldn't try to run the
>>> target
>>> >             version on the
>>> >             >> host, even for sanity tests. Here is the output of the
>>> failure:
>>> >             >>
>>> >             >> $ tail -20
>>> >             >>
>>> >
>>>  /projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/temp/log.do_install.17258
>>> >             >>
>>> >             >> x86_64-diags-linux-ar rc libpython2.7.a
>>> Modules/threadmodule.o
>>> >             >>  Modules/signalmodule.o  Modules/posixmodule.o
>>> Modules/errnomodule.o
>>> >             >>  Modules/pwdmodule.o  Modules/_sre.o
>>> Modules/_codecsmodule.o
>>> >             >>  Modules/_weakref.o  Modules/zipimport.o
>>> Modules/symtablemodule.o
>>> >             >>  Modules/md5module.o Modules/md5.o  Modules/xxsubtype.o
>>> >             >> x86_64-diags-linux-ranlib libpython2.7.a
>>> >             >> Modules/posixmodule.o: In function `posix_tmpnam':
>>> >             >>
>>> >
>>>  /projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Modules/posixmodule.c:7575:
>>> >             >> warning: the use of `tmpnam_r' is dangerous, better use
>>> `mkstemp'
>>> >             >> Modules/posixmodule.o: In function `posix_tempnam':
>>> >             >>
>>> >
>>>  /projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Modules/posixmodule.c:7522:
>>> >             >> warning: the use of `tempnam' is dangerous, better use
>>> `mkstemp'
>>> >             >> x86_64-diags-linux-gcc  -m64 -march=haswell
>>> -mtune=haswell
>>> >             -mfpmath=sse -mavx2
>>> >             >>
>>> --sysroot=/projects/yocto-git/build/tmp/sysroots/continental -Wl,-O1
>>> >             >> -Wl,--hash-style=gnu -Wl,--as-needed -Xlinker
>>> -export-dynamic -o
>>> >             python \
>>> >             >>                         Modules/python.o \
>>> >             >>                         -L. -lpython2.7 -lpthread -ldl
>>> -lpthread
>>> >             -lutil   -lm
>>> >             >>
>>> >
>>>  _PYTHON_PROJECT_BASE=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build
>>> >             >> _PYTHON_HOST_PLATFORM=linux2-x86_64
>>> >             >>
>>> >
>>>  PYTHONPATH=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build/build/lib.linux2-x86_64-2.7:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib/plat-linux2
>>> >             >> python2.7 -S -m sysconfig --generate-posix-vars ;\
>>> >             >>         if test $? -ne 0 ; then \
>>> >             >>                 echo "generate-posix-vars failed" ; \
>>> >             >>                 rm -f ./pybuilddir.txt ; \
>>> >             >>                 exit 1 ; \
>>> >             >>         fi
>>> >             >> Illegal instruction (core dumped)
>>> >             >> make: *** [sharedmods] Error 132
>>> >             >> WARNING: exit code 1 from a shell command.
>>> >             >> ERROR: oe_runmake failed
>>> >             >> ERROR: Function failed: do_install (log file is located
>>> at
>>> >             >>
>>> >
>>>  /projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/temp/log.do_install.17258)
>>> >             >>
>>> >             >> Here is my tune-haswell.inc (ignore some copy/paste
>>> comment
>>> >             issues right now =):
>>> >             >>
>>> >             >> # Settings for the GCC(1) cpu-type "haswell":
>>> >             >> #
>>> >             >> #     Intel Core i7 CPU with 64-bit extensions, MOVBE,
>>> MMX, SSE,
>>> >             SSE2, SSE3,·
>>> >             >> #     SSSE3, SSE4.1, SSE4.2, POPCNT, AVX, AVX2, AES,
>>> PCLMUL,
>>> >             FSGSBASE,
>>> >             >> #     RDRND, FMA, BMI, BMI2 and F16C instruction set
>>> support.
>>> >             >> #
>>> >             >> # This tune is recommended for Intel Nehalem and
>>> Silvermont (e.g.
>>> >             Bay Trail) CPUs
>>> >             >> # (and beyond).
>>> >             >> #
>>> >             >> DEFAULTTUNE ?= "haswell"
>>> >             >>
>>> >             >> # Pull in the previous tune in to pull in
>>> PACKAGE_EXTRA_ARCHS
>>> >             >> require conf/machine/include/tune-corei7.inc
>>> >             >>
>>> >             >> # Extra tune features
>>> >             >> TUNEVALID[haswell] = "Enable haswell specific processor
>>> >             optimizations"
>>> >             >> TUNE_CCARGS .= "${@bb.utils.contains("TUNE_FEATURES",
>>> "haswell", "
>>> >             >> -march=haswell -mtune=haswell -mfpmath=sse -mavx2", "",
>>> d)}"
>>> >             >>
>>> >             >> # Extra tune selections
>>> >             >> AVAILTUNES += "haswell"
>>> >             >> TUNE_FEATURES_tune-haswell =
>>> "${TUNE_FEATURES_tune-x86-64} haswell"
>>> >             >> BASE_LIB_tune-haswell = "lib"
>>> >             >> TUNE_PKGARCH_tune-haswell = "haswell"
>>> >             >> PACKAGE_EXTRA_ARCHS_tune-haswell =
>>> >             "${PACKAGE_EXTRA_ARCHS_tune-corei7} haswell"
>>> >             >>
>>> >             >>
>>> >             >>
>>> >             >
>>> >
>>> >             --
>>> >             _______________________________________________
>>> >             yocto mailing list
>>> >             yocto@yoctoproject.org <mailto:yocto@yoctoproject.org>
>>> >             https://lists.yoctoproject.org/listinfo/yocto
>>> >
>>> >
>>> >
>>> >
>>>
>>>
>>
>

[-- Attachment #2: Type: text/html, Size: 21748 bytes --]

Re: Cross-compiling python fails with new custom tuning

[Mark Hatle]

This is the patch that we ended up using:

http://lists.openembedded.org/pipermail/openembedded-core/2015-November/113457.html

http://lists.openembedded.org/pipermail/openembedded-core/2015-November/113456.html

http://lists.openembedded.org/pipermail/openembedded-core/2015-November/113458.html

On 11/30/15 5:12 PM, Michael Habibi wrote:
> OK I've dug into this further today, and here are some findings:
> 
> With my patch (which is similar to yours), it fails do_compile/package_qa
> because of poisoned system directories during the compilation steps. This
> approach seems to be a bust. Originally when I tested it, I hacked up the
> Makefile and only re-ran do_install onward, and didn't re-run do_compile
> (because I had to create the patch). Once I created the patch and re-ran from
> clean, I got some errors.
> 
> It appears the only reason it is using that directory in PYTHONPATH at all (the
> one generated from the shell command and pybuilddir.txt) is for a potential
> sysconfig file in there. I tracked it down to a comment in a bug on Python's bug
> tracker:
> 
> http://bugs.python.org/issue15484#msg180577
> 
> If we change that directory, it will pull in the sysconfig settings from the
> native Python configure (probably defaults to native sysconfig settings because
> the PATH doesn't include the overrides), and then do some really bad things. The
> question then became: in the unmodified build, why does do_install fail but
> do_compile doesn't? They both seem to be using sharedmods recipe which uses
> $PYTHONPATH. The answer is that when we run do_compile the first time around,
> the offending *.so's are not generated yet, and thus are not included by
> python-native. The second time around (in do_install), the *.so's are in the
> path and so python-native will use them and override its own.
> 
> My latest experiment is to remove that location altogether from PYTHONPATH and
> see if it works (neither the shell-generated directory in python-cross, nor the
> lib-dynload directory from python-native). The compile works the first time
> through despite there being no sysconfig data, so I'm wondering if the install
> step will also work without finding any data. python-native should continue to
> work with its internal settings to find modules and libraries.
> 
> To be continued tomorrow, I'm sure...
> 
> On Mon, Nov 30, 2015 at 11:37 AM, Michael Habibi <mikehabibi@gmail.com
> <mailto:mikehabibi@gmail.com>> wrote:
> 
>     Mark, I am continuing to look at this. I hope you don't mind if I keep
>     updating this thread with my investigation.
> 
>     First, I figured out my confusion. I was associating the wrong Makefile
>     command with the illegal instruction. It was actually the command under the
>     recipe for 'sharedmods', but it was silenced in the Makefile so I assumed it
>     was another line. You're right that the path with the *.so's is the
>     offending path. I've scoured online patches and can't understand how this
>     mechanism works outside of Yocto. I figure people trying to compile for ARM
>     would have similar issues. There seems to be a fundamental problem with how
>     Python sets up PYTHONPATH using the pybuilddir file and $abs_builddir.
>     However, other people seem to have successfully built python for other
>     architectures, so I'm still confused.
> 
>     I hardcoded a fix for PYTHONPATH to use the host lib-dynload directory, and
>     the python recipe successfully passed. Can you elaborate on the issues you
>     were having with your patch? The only difference is that I maintained the
>     other two directories in $PYTHONPATH,
>     e.g. $(srcdir)/Lib:$(srcdir)/Lib/plat-$(MACHDEP), and only replaced the
>     offending one.
> 
>     On Wed, Nov 25, 2015 at 4:58 PM, Michael Habibi <mikehabibi@gmail.com
>     <mailto:mikehabibi@gmail.com>> wrote:
> 
>         Mark,
> 
>         I ran the same command, including the offending PYTHONPATH, from the
>         shell without error. Any reason why that would work, despite it not
>         working from within the Makefile? I was thinking it was another
>         environment variable that was causing the issue (one that's not set in
>         my bash environment by default). My $PATH didn't have python-native so I
>         called it using an absolute path, but otherwise it's the same command
>         run from the Makefile (parsed, of course):
> 
>         bash$
>         _PYTHON_PROJECT_BASE=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build
>         _PYTHON_HOST_PLATFORM=linux2-x86_64
>         PYTHONPATH=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build/build/lib.linux2-x86_64-2.7:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib/plat-linux2
>         /projects/yocto-git/build/tmp/sysroots/x86_64-linux/usr/bin/python-native/python2.7
>         -S -m sysconfig --generate-posix-vars
>         bash$ ls pybuilddir.txt
>         pybuilddir.txt
> 
>         Maybe I'm missing something.
> 
> 
>         On Wed, Nov 25, 2015 at 4:38 PM, Mark Hatle <mark.hatle@windriver.com
>         <mailto:mark.hatle@windriver.com>> wrote:
> 
>             On 11/25/15 10:11 AM, Michael Habibi wrote:
>             > Well, I'm wrong yet again. A 'which python2.7' shows that it's pointing to a
>             > native version of Python:
>             >
>             > *| which python2.7*
>             > *|
>             >
>             /projects/yocto-git/build/tmp/sysroots/x86_64-linux/usr/bin/python-native/python2.7*
>             > |
>             > _PYTHON_PROJECT_BASE=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build
>             > _PYTHON_HOST_PLATFORM=linux2-x86_64
>             > PYTHONPATH=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build/build/lib.linux2-x86_64-2.7:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib/plat-linux2
>             > python2.7 -S -m sysconfig --generate-posix-vars ;\
>             > |       if test $? -ne 0 ; then \
>             > |               echo "generate-posix-vars failed" ; \
>             > |               rm -f ./pybuilddir.txt ; \
>             > |               exit 1 ; \
>             > |       fi
>             > | Illegal instruction (core dumped)
>             > | make: *** [sharedmods] Error 132
>             >
>             > I guess one of the environmental variables being passed in is screwing up what
>             > it's doing. Unfortunately I don't know enough about the inner workings of Python
>             > to be of much help moving forward. I gave it my best shot!
> 
>             The problem I tracked down was:
> 
>             PYTHONPATH=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build/build/lib.linux2-x86_64-2.7
> 
>             This is full of .so objects when loaded cause the illegal instruction.
> 
>             I came up with a patch that I thought was going to fix it, but it's
>             triggered
>             other failures.
> 
>             http://lists.openembedded.org/pipermail/openembedded-core/2015-November/113341.html
> 
>             Just to be clear.. this does NOT work properly in many cases.  But
>             might give
>             you or someone else a clue as to how to possibly fix it.
> 
>             --Mark
> 
>             >
>             > On Wed, Nov 25, 2015 at 10:01 AM, Michael Habibi <mikehabibi@gmail.com <mailto:mikehabibi@gmail.com>
>             > <mailto:mikehabibi@gmail.com <mailto:mikehabibi@gmail.com>>> wrote:
>             >
>             >     Ah sorry, I misspoke. I walked through the Makefile and configure scripts a
>             >     bit more, and $(PYTHON_FOR_BUILD) should in fact be a host version of
>             >     Python. It seems it's not being configured correctly. Either the path is
>             >     setup wrong and python2.7 is not pointing to the right version, or python2.7
>             >     needs to be pointing to the absolute path instead of relying on $PATH.
>             >
>             >
>             >
>             >
>             >     On Wed, Nov 25, 2015 at 9:45 AM, Michael Habibi <mikehabibi@gmail.com <mailto:mikehabibi@gmail.com>
>             >     <mailto:mikehabibi@gmail.com <mailto:mikehabibi@gmail.com>>>
>             wrote:
>             >
>             >         For what it's worth, this is the offending portion of
>             Makefile.pre:452:
>             >
>             >         # Create build directory and generate the sysconfig
>             build-time data there.
>             >         # pybuilddir.txt contains the name of the build dir and is
>             used for
>             >         # sys.path fixup -- see Modules/getpath.c.
>             >         # Since this step runs before shared modules are built,
>             try to avoid
>             >         bootstrap
>             >         # problems by creating a dummy pybuilddir.txt just to
>             allow interpreter
>             >         # initialization to succeed.  It will be overwritten by
>             generate-posix-vars
>             >         # or removed in case of failure.
>             >         pybuilddir.txt: $(BUILDPYTHON)
>             >         → @echo "none" > ./pybuilddir.txt
>             >         → $(RUNSHARED) $(PYTHON_FOR_BUILD) -S -m sysconfig
>             --generate-posix-vars ;\
>             >         → if test $$? -ne 0 ; then \
>             >         → → echo "generate-posix-vars failed" ; \
>             >         → → rm -f ./pybuilddir.txt ; \
>             >         → → exit 1 ; \
>             >         → fi
>             >
>             >         I don't know enough about the Python build to understand
>             what it's
>             >         trying to do, but perhaps replacing PYTHON_FOR_BUILD with
>             HOSTPYTHON may
>             >         be acceptable?
>             >
>             >         I'm surprised this seems to work for other people, since
>             this should be
>             >         failing for anyone using Python on a target platform
>             different from
>             >         their host.
>             >
>             >         On Wed, Nov 25, 2015 at 9:02 AM, Mark Hatle
>             <mark.hatle@windriver.com <mailto:mark.hatle@windriver.com>
>             >         <mailto:mark.hatle@windriver.com
>             <mailto:mark.hatle@windriver.com>>> wrote:
>             >
>             >             On 11/24/15 3:23 PM, Mark Hatle wrote:
>             >             > My guess is that there is a bug in the python
>             integration where
>             >             it's not
>             >             > realizing the host and target are different systems,
>             so it's
>             >             trying to run a
>             >             > target program on your host.  Your host isn't
>             haswell, so...
>             >             Illegal Instruction
>             >             > -- and the system stops.
>             >
>             >             Just an FYI, I hit the same problem today.  I suspect
>             it is the host
>             >             trying to
>             >             run target software and I'm looking into it.
>             >
>             >             --Mark
>             >
>             >             > The alternative would be something is running QEMU
>             to execute a
>             >             target binary
>             >             > and QEMU doesn't have instruction support -- but
>             that doesn't look
>             >             like the case
>             >             > here.
>             >             >
>             >             > --Mark
>             >             >
>             >             > On 11/24/15 3:06 PM, Michael Habibi wrote:
>             >             >> All,
>             >             >>
>             >             >> I added a new machine definition with tuning
>             parameters for haswell
>             >             >> microarchitectures (basically just duplicated
>             corei7 but tuned it
>             >             for haswell).
>             >             >> This seems to work correctly, but failed when
>             running do_install
>             >             for python
>             >             >> toward the end of the build process. I am running
>             with the Yocto
>             >             2.0 framework,
>             >             >> with very minimal changes to create a new
>             distribution and
>             >             machine for our
>             >             >> custom embedded device. Note I had this distro
>             configuration
>             >             working before, and
>             >             >> the only difference is I added a new machine with
>             this tuning.
>             >             >>
>             >             >> I believe the issue is because, as part of the
>             do_install step
>             >             for Python, it
>             >             >> attempts to run python on the local host, despite being
>             >             cross-compiled. However,
>             >             >> it is tuned for a processor that my host machine
>             doesn't run, so
>             >             I get an
>             >             >> instruction exception.
>             >             >>
>             >             >> Did I do something weird? I figure python would be
>             configured for
>             >             >> cross-compiling and therefore wouldn't try to run
>             the target
>             >             version on the
>             >             >> host, even for sanity tests. Here is the output of
>             the failure:
>             >             >>
>             >             >> $ tail -20
>             >             >>
>             >           
>              /projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/temp/log.do_install.17258
>             >             >>
>             >             >> x86_64-diags-linux-ar rc libpython2.7.a
>             Modules/threadmodule.o
>             >             >>  Modules/signalmodule.o  Modules/posixmodule.o 
>             Modules/errnomodule.o
>             >             >>  Modules/pwdmodule.o  Modules/_sre.o 
>             Modules/_codecsmodule.o
>             >             >>  Modules/_weakref.o  Modules/zipimport.o 
>             Modules/symtablemodule.o
>             >             >>  Modules/md5module.o Modules/md5.o  Modules/xxsubtype.o
>             >             >> x86_64-diags-linux-ranlib libpython2.7.a
>             >             >> Modules/posixmodule.o: In function `posix_tmpnam':
>             >             >>
>             >           
>              /projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Modules/posixmodule.c:7575:
>             >             >> warning: the use of `tmpnam_r' is dangerous, better
>             use `mkstemp'
>             >             >> Modules/posixmodule.o: In function `posix_tempnam':
>             >             >>
>             >           
>              /projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Modules/posixmodule.c:7522:
>             >             >> warning: the use of `tempnam' is dangerous, better
>             use `mkstemp'
>             >             >> x86_64-diags-linux-gcc  -m64 -march=haswell
>             -mtune=haswell
>             >             -mfpmath=sse -mavx2
>             >             >>
>             --sysroot=/projects/yocto-git/build/tmp/sysroots/continental -Wl,-O1
>             >             >> -Wl,--hash-style=gnu -Wl,--as-needed -Xlinker
>             -export-dynamic -o
>             >             python \
>             >             >>                         Modules/python.o \
>             >             >>                         -L. -lpython2.7 -lpthread
>             -ldl  -lpthread
>             >             -lutil   -lm
>             >             >>
>             >           
>              _PYTHON_PROJECT_BASE=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build
>             >             >> _PYTHON_HOST_PLATFORM=linux2-x86_64
>             >             >>
>             >           
>              PYTHONPATH=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build/build/lib.linux2-x86_64-2.7:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib/plat-linux2
>             >             >> python2.7 -S -m sysconfig --generate-posix-vars ;\
>             >             >>         if test $? -ne 0 ; then \
>             >             >>                 echo "generate-posix-vars failed" ; \
>             >             >>                 rm -f ./pybuilddir.txt ; \
>             >             >>                 exit 1 ; \
>             >             >>         fi
>             >             >> Illegal instruction (core dumped)
>             >             >> make: *** [sharedmods] Error 132
>             >             >> WARNING: exit code 1 from a shell command.
>             >             >> ERROR: oe_runmake failed
>             >             >> ERROR: Function failed: do_install (log file is
>             located at
>             >             >>
>             >           
>              /projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/temp/log.do_install.17258)
>             >             >>
>             >             >> Here is my tune-haswell.inc (ignore some copy/paste
>             comment
>             >             issues right now =):
>             >             >>
>             >             >> # Settings for the GCC(1) cpu-type "haswell":
>             >             >> #
>             >             >> #     Intel Core i7 CPU with 64-bit extensions,
>             MOVBE, MMX, SSE,
>             >             SSE2, SSE3,·
>             >             >> #     SSSE3, SSE4.1, SSE4.2, POPCNT, AVX, AVX2,
>             AES, PCLMUL,
>             >             FSGSBASE,
>             >             >> #     RDRND, FMA, BMI, BMI2 and F16C instruction
>             set support.
>             >             >> #
>             >             >> # This tune is recommended for Intel Nehalem and
>             Silvermont (e.g.
>             >             Bay Trail) CPUs
>             >             >> # (and beyond).
>             >             >> #
>             >             >> DEFAULTTUNE ?= "haswell"
>             >             >>
>             >             >> # Pull in the previous tune in to pull in
>             PACKAGE_EXTRA_ARCHS
>             >             >> require conf/machine/include/tune-corei7.inc
>             >             >>
>             >             >> # Extra tune features
>             >             >> TUNEVALID[haswell] = "Enable haswell specific processor
>             >             optimizations"
>             >             >> TUNE_CCARGS .=
>             "${@bb.utils.contains("TUNE_FEATURES", "haswell", "
>             >             >> -march=haswell -mtune=haswell -mfpmath=sse -mavx2",
>             "", d)}"
>             >             >>
>             >             >> # Extra tune selections
>             >             >> AVAILTUNES += "haswell"
>             >             >> TUNE_FEATURES_tune-haswell =
>             "${TUNE_FEATURES_tune-x86-64} haswell"
>             >             >> BASE_LIB_tune-haswell = "lib"
>             >             >> TUNE_PKGARCH_tune-haswell = "haswell"
>             >             >> PACKAGE_EXTRA_ARCHS_tune-haswell =
>             >             "${PACKAGE_EXTRA_ARCHS_tune-corei7} haswell"
>             >             >>
>             >             >>
>             >             >>
>             >             >
>             >
>             >             --
>             >             _______________________________________________
>             >             yocto mailing list
>             >             yocto@yoctoproject.org <mailto:yocto@yoctoproject.org>
>             <mailto:yocto@yoctoproject.org <mailto:yocto@yoctoproject.org>>
>             >             https://lists.yoctoproject.org/listinfo/yocto
>             >
>             >
>             >
>             >
> 
> 
> 
> 

Re: Cross-compiling python fails with new custom tuning

[Michael Habibi]

[-- Attachment #1: Type: text/plain, Size: 22713 bytes --]

Mark, great! I was literally about to try double-stacking the PATH next
(since removing both didn't work for reasons that became obvious after
running the experiment), so that it would grab the *.so's from
python-native and then the sysconfig from the python-cross. Looks like I
may need to subscribe to that list so I'm aware of these patches!

I'm not sure what the streams look like (I can't visualize it yet), but is
this change in any Yocto branch yet, or does it need to get pulled by
someone? I can merge it locally, but just curious how the OE-Core and Yocto
repos are related.

On Mon, Nov 30, 2015 at 6:07 PM, Mark Hatle <mark.hatle@windriver.com>
wrote:

> This is the patch that we ended up using:
>
>
> http://lists.openembedded.org/pipermail/openembedded-core/2015-November/113457.html
>
>
> http://lists.openembedded.org/pipermail/openembedded-core/2015-November/113456.html
>
>
> http://lists.openembedded.org/pipermail/openembedded-core/2015-November/113458.html
>
> On 11/30/15 5:12 PM, Michael Habibi wrote:
> > OK I've dug into this further today, and here are some findings:
> >
> > With my patch (which is similar to yours), it fails do_compile/package_qa
> > because of poisoned system directories during the compilation steps. This
> > approach seems to be a bust. Originally when I tested it, I hacked up the
> > Makefile and only re-ran do_install onward, and didn't re-run do_compile
> > (because I had to create the patch). Once I created the patch and re-ran
> from
> > clean, I got some errors.
> >
> > It appears the only reason it is using that directory in PYTHONPATH at
> all (the
> > one generated from the shell command and pybuilddir.txt) is for a
> potential
> > sysconfig file in there. I tracked it down to a comment in a bug on
> Python's bug
> > tracker:
> >
> > http://bugs.python.org/issue15484#msg180577
> >
> > If we change that directory, it will pull in the sysconfig settings from
> the
> > native Python configure (probably defaults to native sysconfig settings
> because
> > the PATH doesn't include the overrides), and then do some really bad
> things. The
> > question then became: in the unmodified build, why does do_install fail
> but
> > do_compile doesn't? They both seem to be using sharedmods recipe which
> uses
> > $PYTHONPATH. The answer is that when we run do_compile the first time
> around,
> > the offending *.so's are not generated yet, and thus are not included by
> > python-native. The second time around (in do_install), the *.so's are in
> the
> > path and so python-native will use them and override its own.
> >
> > My latest experiment is to remove that location altogether from
> PYTHONPATH and
> > see if it works (neither the shell-generated directory in python-cross,
> nor the
> > lib-dynload directory from python-native). The compile works the first
> time
> > through despite there being no sysconfig data, so I'm wondering if the
> install
> > step will also work without finding any data. python-native should
> continue to
> > work with its internal settings to find modules and libraries.
> >
> > To be continued tomorrow, I'm sure...
> >
> > On Mon, Nov 30, 2015 at 11:37 AM, Michael Habibi <mikehabibi@gmail.com
> > <mailto:mikehabibi@gmail.com>> wrote:
> >
> >     Mark, I am continuing to look at this. I hope you don't mind if I
> keep
> >     updating this thread with my investigation.
> >
> >     First, I figured out my confusion. I was associating the wrong
> Makefile
> >     command with the illegal instruction. It was actually the command
> under the
> >     recipe for 'sharedmods', but it was silenced in the Makefile so I
> assumed it
> >     was another line. You're right that the path with the *.so's is the
> >     offending path. I've scoured online patches and can't understand how
> this
> >     mechanism works outside of Yocto. I figure people trying to compile
> for ARM
> >     would have similar issues. There seems to be a fundamental problem
> with how
> >     Python sets up PYTHONPATH using the pybuilddir file and
> $abs_builddir.
> >     However, other people seem to have successfully built python for
> other
> >     architectures, so I'm still confused.
> >
> >     I hardcoded a fix for PYTHONPATH to use the host lib-dynload
> directory, and
> >     the python recipe successfully passed. Can you elaborate on the
> issues you
> >     were having with your patch? The only difference is that I
> maintained the
> >     other two directories in $PYTHONPATH,
> >     e.g. $(srcdir)/Lib:$(srcdir)/Lib/plat-$(MACHDEP), and only replaced
> the
> >     offending one.
> >
> >     On Wed, Nov 25, 2015 at 4:58 PM, Michael Habibi <
> mikehabibi@gmail.com
> >     <mailto:mikehabibi@gmail.com>> wrote:
> >
> >         Mark,
> >
> >         I ran the same command, including the offending PYTHONPATH, from
> the
> >         shell without error. Any reason why that would work, despite it
> not
> >         working from within the Makefile? I was thinking it was another
> >         environment variable that was causing the issue (one that's not
> set in
> >         my bash environment by default). My $PATH didn't have
> python-native so I
> >         called it using an absolute path, but otherwise it's the same
> command
> >         run from the Makefile (parsed, of course):
> >
> >         bash$
> >
>  _PYTHON_PROJECT_BASE=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build
> >         _PYTHON_HOST_PLATFORM=linux2-x86_64
> >
>  PYTHONPATH=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build/build/lib.linux2-x86_64-2.7:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib/plat-linux2
> >
>  /projects/yocto-git/build/tmp/sysroots/x86_64-linux/usr/bin/python-native/python2.7
> >         -S -m sysconfig --generate-posix-vars
> >         bash$ ls pybuilddir.txt
> >         pybuilddir.txt
> >
> >         Maybe I'm missing something.
> >
> >
> >         On Wed, Nov 25, 2015 at 4:38 PM, Mark Hatle <
> mark.hatle@windriver.com
> >         <mailto:mark.hatle@windriver.com>> wrote:
> >
> >             On 11/25/15 10:11 AM, Michael Habibi wrote:
> >             > Well, I'm wrong yet again. A 'which python2.7' shows that
> it's pointing to a
> >             > native version of Python:
> >             >
> >             > *| which python2.7*
> >             > *|
> >             >
> >
>  /projects/yocto-git/build/tmp/sysroots/x86_64-linux/usr/bin/python-native/python2.7*
> >             > |
> >             >
> _PYTHON_PROJECT_BASE=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build
> >             > _PYTHON_HOST_PLATFORM=linux2-x86_64
> >             >
> PYTHONPATH=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build/build/lib.linux2-x86_64-2.7:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib/plat-linux2
> >             > python2.7 -S -m sysconfig --generate-posix-vars ;\
> >             > |       if test $? -ne 0 ; then \
> >             > |               echo "generate-posix-vars failed" ; \
> >             > |               rm -f ./pybuilddir.txt ; \
> >             > |               exit 1 ; \
> >             > |       fi
> >             > | Illegal instruction (core dumped)
> >             > | make: *** [sharedmods] Error 132
> >             >
> >             > I guess one of the environmental variables being passed in
> is screwing up what
> >             > it's doing. Unfortunately I don't know enough about the
> inner workings of Python
> >             > to be of much help moving forward. I gave it my best shot!
> >
> >             The problem I tracked down was:
> >
> >
>  PYTHONPATH=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build/build/lib.linux2-x86_64-2.7
> >
> >             This is full of .so objects when loaded cause the illegal
> instruction.
> >
> >             I came up with a patch that I thought was going to fix it,
> but it's
> >             triggered
> >             other failures.
> >
> >
> http://lists.openembedded.org/pipermail/openembedded-core/2015-November/113341.html
> >
> >             Just to be clear.. this does NOT work properly in many
> cases.  But
> >             might give
> >             you or someone else a clue as to how to possibly fix it.
> >
> >             --Mark
> >
> >             >
> >             > On Wed, Nov 25, 2015 at 10:01 AM, Michael Habibi <
> mikehabibi@gmail.com <mailto:mikehabibi@gmail.com>
> >             > <mailto:mikehabibi@gmail.com <mailto:mikehabibi@gmail.com>>>
> wrote:
> >             >
> >             >     Ah sorry, I misspoke. I walked through the Makefile
> and configure scripts a
> >             >     bit more, and $(PYTHON_FOR_BUILD) should in fact be a
> host version of
> >             >     Python. It seems it's not being configured correctly.
> Either the path is
> >             >     setup wrong and python2.7 is not pointing to the right
> version, or python2.7
> >             >     needs to be pointing to the absolute path instead of
> relying on $PATH.
> >             >
> >             >
> >             >
> >             >
> >             >     On Wed, Nov 25, 2015 at 9:45 AM, Michael Habibi <
> mikehabibi@gmail.com <mailto:mikehabibi@gmail.com>
> >             >     <mailto:mikehabibi@gmail.com <mailto:
> mikehabibi@gmail.com>>>
> >             wrote:
> >             >
> >             >         For what it's worth, this is the offending portion
> of
> >             Makefile.pre:452:
> >             >
> >             >         # Create build directory and generate the sysconfig
> >             build-time data there.
> >             >         # pybuilddir.txt contains the name of the build
> dir and is
> >             used for
> >             >         # sys.path fixup -- see Modules/getpath.c.
> >             >         # Since this step runs before shared modules are
> built,
> >             try to avoid
> >             >         bootstrap
> >             >         # problems by creating a dummy pybuilddir.txt just
> to
> >             allow interpreter
> >             >         # initialization to succeed.  It will be
> overwritten by
> >             generate-posix-vars
> >             >         # or removed in case of failure.
> >             >         pybuilddir.txt: $(BUILDPYTHON)
> >             >         → @echo "none" > ./pybuilddir.txt
> >             >         → $(RUNSHARED) $(PYTHON_FOR_BUILD) -S -m sysconfig
> >             --generate-posix-vars ;\
> >             >         → if test $$? -ne 0 ; then \
> >             >         → → echo "generate-posix-vars failed" ; \
> >             >         → → rm -f ./pybuilddir.txt ; \
> >             >         → → exit 1 ; \
> >             >         → fi
> >             >
> >             >         I don't know enough about the Python build to
> understand
> >             what it's
> >             >         trying to do, but perhaps replacing
> PYTHON_FOR_BUILD with
> >             HOSTPYTHON may
> >             >         be acceptable?
> >             >
> >             >         I'm surprised this seems to work for other people,
> since
> >             this should be
> >             >         failing for anyone using Python on a target
> platform
> >             different from
> >             >         their host.
> >             >
> >             >         On Wed, Nov 25, 2015 at 9:02 AM, Mark Hatle
> >             <mark.hatle@windriver.com <mailto:mark.hatle@windriver.com>
> >             >         <mailto:mark.hatle@windriver.com
> >             <mailto:mark.hatle@windriver.com>>> wrote:
> >             >
> >             >             On 11/24/15 3:23 PM, Mark Hatle wrote:
> >             >             > My guess is that there is a bug in the python
> >             integration where
> >             >             it's not
> >             >             > realizing the host and target are different
> systems,
> >             so it's
> >             >             trying to run a
> >             >             > target program on your host.  Your host isn't
> >             haswell, so...
> >             >             Illegal Instruction
> >             >             > -- and the system stops.
> >             >
> >             >             Just an FYI, I hit the same problem today.  I
> suspect
> >             it is the host
> >             >             trying to
> >             >             run target software and I'm looking into it.
> >             >
> >             >             --Mark
> >             >
> >             >             > The alternative would be something is
> running QEMU
> >             to execute a
> >             >             target binary
> >             >             > and QEMU doesn't have instruction support --
> but
> >             that doesn't look
> >             >             like the case
> >             >             > here.
> >             >             >
> >             >             > --Mark
> >             >             >
> >             >             > On 11/24/15 3:06 PM, Michael Habibi wrote:
> >             >             >> All,
> >             >             >>
> >             >             >> I added a new machine definition with tuning
> >             parameters for haswell
> >             >             >> microarchitectures (basically just
> duplicated
> >             corei7 but tuned it
> >             >             for haswell).
> >             >             >> This seems to work correctly, but failed
> when
> >             running do_install
> >             >             for python
> >             >             >> toward the end of the build process. I am
> running
> >             with the Yocto
> >             >             2.0 framework,
> >             >             >> with very minimal changes to create a new
> >             distribution and
> >             >             machine for our
> >             >             >> custom embedded device. Note I had this
> distro
> >             configuration
> >             >             working before, and
> >             >             >> the only difference is I added a new
> machine with
> >             this tuning.
> >             >             >>
> >             >             >> I believe the issue is because, as part of
> the
> >             do_install step
> >             >             for Python, it
> >             >             >> attempts to run python on the local host,
> despite being
> >             >             cross-compiled. However,
> >             >             >> it is tuned for a processor that my host
> machine
> >             doesn't run, so
> >             >             I get an
> >             >             >> instruction exception.
> >             >             >>
> >             >             >> Did I do something weird? I figure python
> would be
> >             configured for
> >             >             >> cross-compiling and therefore wouldn't try
> to run
> >             the target
> >             >             version on the
> >             >             >> host, even for sanity tests. Here is the
> output of
> >             the failure:
> >             >             >>
> >             >             >> $ tail -20
> >             >             >>
> >             >
> >
> /projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/temp/log.do_install.17258
> >             >             >>
> >             >             >> x86_64-diags-linux-ar rc libpython2.7.a
> >             Modules/threadmodule.o
> >             >             >>  Modules/signalmodule.o
> Modules/posixmodule.o
> >             Modules/errnomodule.o
> >             >             >>  Modules/pwdmodule.o  Modules/_sre.o
> >             Modules/_codecsmodule.o
> >             >             >>  Modules/_weakref.o  Modules/zipimport.o
> >             Modules/symtablemodule.o
> >             >             >>  Modules/md5module.o Modules/md5.o
> Modules/xxsubtype.o
> >             >             >> x86_64-diags-linux-ranlib libpython2.7.a
> >             >             >> Modules/posixmodule.o: In function
> `posix_tmpnam':
> >             >             >>
> >             >
> >
> /projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Modules/posixmodule.c:7575:
> >             >             >> warning: the use of `tmpnam_r' is
> dangerous, better
> >             use `mkstemp'
> >             >             >> Modules/posixmodule.o: In function
> `posix_tempnam':
> >             >             >>
> >             >
> >
> /projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Modules/posixmodule.c:7522:
> >             >             >> warning: the use of `tempnam' is dangerous,
> better
> >             use `mkstemp'
> >             >             >> x86_64-diags-linux-gcc  -m64 -march=haswell
> >             -mtune=haswell
> >             >             -mfpmath=sse -mavx2
> >             >             >>
> >             --sysroot=/projects/yocto-git/build/tmp/sysroots/continental
> -Wl,-O1
> >             >             >> -Wl,--hash-style=gnu -Wl,--as-needed
> -Xlinker
> >             -export-dynamic -o
> >             >             python \
> >             >             >>                         Modules/python.o \
> >             >             >>                         -L. -lpython2.7
> -lpthread
> >             -ldl  -lpthread
> >             >             -lutil   -lm
> >             >             >>
> >             >
> >
> _PYTHON_PROJECT_BASE=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build
> >             >             >> _PYTHON_HOST_PLATFORM=linux2-x86_64
> >             >             >>
> >             >
> >
> PYTHONPATH=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build/build/lib.linux2-x86_64-2.7:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib/plat-linux2
> >             >             >> python2.7 -S -m sysconfig
> --generate-posix-vars ;\
> >             >             >>         if test $? -ne 0 ; then \
> >             >             >>                 echo "generate-posix-vars
> failed" ; \
> >             >             >>                 rm -f ./pybuilddir.txt ; \
> >             >             >>                 exit 1 ; \
> >             >             >>         fi
> >             >             >> Illegal instruction (core dumped)
> >             >             >> make: *** [sharedmods] Error 132
> >             >             >> WARNING: exit code 1 from a shell command.
> >             >             >> ERROR: oe_runmake failed
> >             >             >> ERROR: Function failed: do_install (log
> file is
> >             located at
> >             >             >>
> >             >
> >
> /projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/temp/log.do_install.17258)
> >             >             >>
> >             >             >> Here is my tune-haswell.inc (ignore some
> copy/paste
> >             comment
> >             >             issues right now =):
> >             >             >>
> >             >             >> # Settings for the GCC(1) cpu-type
> "haswell":
> >             >             >> #
> >             >             >> #     Intel Core i7 CPU with 64-bit
> extensions,
> >             MOVBE, MMX, SSE,
> >             >             SSE2, SSE3,·
> >             >             >> #     SSSE3, SSE4.1, SSE4.2, POPCNT, AVX,
> AVX2,
> >             AES, PCLMUL,
> >             >             FSGSBASE,
> >             >             >> #     RDRND, FMA, BMI, BMI2 and F16C
> instruction
> >             set support.
> >             >             >> #
> >             >             >> # This tune is recommended for Intel
> Nehalem and
> >             Silvermont (e.g.
> >             >             Bay Trail) CPUs
> >             >             >> # (and beyond).
> >             >             >> #
> >             >             >> DEFAULTTUNE ?= "haswell"
> >             >             >>
> >             >             >> # Pull in the previous tune in to pull in
> >             PACKAGE_EXTRA_ARCHS
> >             >             >> require conf/machine/include/tune-corei7.inc
> >             >             >>
> >             >             >> # Extra tune features
> >             >             >> TUNEVALID[haswell] = "Enable haswell
> specific processor
> >             >             optimizations"
> >             >             >> TUNE_CCARGS .=
> >             "${@bb.utils.contains("TUNE_FEATURES", "haswell", "
> >             >             >> -march=haswell -mtune=haswell -mfpmath=sse
> -mavx2",
> >             "", d)}"
> >             >             >>
> >             >             >> # Extra tune selections
> >             >             >> AVAILTUNES += "haswell"
> >             >             >> TUNE_FEATURES_tune-haswell =
> >             "${TUNE_FEATURES_tune-x86-64} haswell"
> >             >             >> BASE_LIB_tune-haswell = "lib"
> >             >             >> TUNE_PKGARCH_tune-haswell = "haswell"
> >             >             >> PACKAGE_EXTRA_ARCHS_tune-haswell =
> >             >             "${PACKAGE_EXTRA_ARCHS_tune-corei7} haswell"
> >             >             >>
> >             >             >>
> >             >             >>
> >             >             >
> >             >
> >             >             --
> >             >             _______________________________________________
> >             >             yocto mailing list
> >             >             yocto@yoctoproject.org <mailto:
> yocto@yoctoproject.org>
> >             <mailto:yocto@yoctoproject.org <mailto:
> yocto@yoctoproject.org>>
> >             >             https://lists.yoctoproject.org/listinfo/yocto
> >             >
> >             >
> >             >
> >             >
> >
> >
> >
> >
>
>

[-- Attachment #2: Type: text/html, Size: 31615 bytes --]