All of lore.kernel.org
 help / color / mirror / Atom feed
* [Buildroot] [Bug 7478] New: Multiple chosen python modules are not built due to missing header files in include path
@ 2014-09-27 22:49 bugzilla at busybox.net
  2014-09-27 22:50 ` [Buildroot] [Bug 7478] " bugzilla at busybox.net
                   ` (12 more replies)
  0 siblings, 13 replies; 14+ messages in thread
From: bugzilla at busybox.net @ 2014-09-27 22:49 UTC (permalink / raw)
  To: buildroot

https://bugs.busybox.net/show_bug.cgi?id=7478

           Summary: Multiple chosen python modules are not built due to
                    missing header files in include path
           Product: buildroot
           Version: 2014.08
          Platform: PC
        OS/Version: Linux
            Status: NEW
          Severity: normal
          Priority: P5
         Component: Other
        AssignedTo: unassigned at buildroot.uclibc.org
        ReportedBy: cma at vtxnet.ch
                CC: buildroot at uclibc.org
   Estimated Hours: 0.0


Python doesn't build some of the requested modules because the setup.py script
is unable to locate the respective header files. The affected modules are
sqlite3, ssl, zlib and bsddb. The build completes successfully and the target
images are correctly generated, but these modules will not work in python.

Buildroot target is i686. Buildroot version is 2014.11-git-00330-g6712213.

Chosen modules in .config:
BR2_PACKAGE_PYTHON_BZIP2=y
BR2_PACKAGE_PYTHON_BSDDB=y
BR2_PACKAGE_PYTHON_CODECSCJK=y
BR2_PACKAGE_PYTHON_CURSES=y
BR2_PACKAGE_PYTHON_PYEXPAT=y
BR2_PACKAGE_PYTHON_READLINE=y
BR2_PACKAGE_PYTHON_SSL=y
BR2_PACKAGE_PYTHON_UNICODEDATA=y
BR2_PACKAGE_PYTHON_UCS2=y
# BR2_PACKAGE_PYTHON_UCS4 is not set
BR2_PACKAGE_PYTHON_SQLITE=y
BR2_PACKAGE_PYTHON_ZLIB=y
BR2_PACKAGE_PYTHON_HASHLIB=y

BR2_PACKAGE_SQLITE=y
# BR2_PACKAGE_SQLITE_READLINE is not set
# BR2_PACKAGE_SQLITE_STAT3 is not set
# BR2_PACKAGE_SQLITE_ENABLE_FTS3 is not set
# BR2_PACKAGE_SQLITE_ENABLE_UNLOCK_NOTIFY is not set
# BR2_PACKAGE_SQLITE_SECURE_DELETE is not set
# BR2_PACKAGE_SQLITE_NO_SYNC is not set


Relevant build log snippet (python-2.7.8-build.log:2878):
Python build finished, but the necessary bits to build these modules were not
found:
_bsddb             _sqlite3           _ssl            
_tkinter           bsddb185           dbm             
dl                 gdbm               imageop         
linuxaudiodev      nis                ossaudiodev     
sunaudiodev        zlib                     
To find the necessary bits, look in setup.py in detect_modules() for the
module's name.


After booting the target, importing these modules will result in an
ImportError:
# python -c "import sqlite3"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python2.7/sqlite3/__init__.py", line 24, in <module>
    from dbapi2 import *
  File "/usr/lib/python2.7/sqlite3/dbapi2.py", line 28, in <module>
    from _sqlite3 import *
ImportError: No module named _sqlite3

Note: sqlite3 library is installed on the target and the sqlite3 shell is
working as excepted.

# python -c "import ssl"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python2.7/ssl.py", line 60, in <module>
    import _ssl             # if we can't import it, let the error propagate
ImportError: No module named _ssl


As a temporary workaround, copying the sqlite header file in the python include
directory before building python permits the module compilation (here for
sqlite): 
$ cp output/build/sqlite-3080500/sqlite3.h output/build/python-2.7.8/Include/
For the other modules, I haven't tested.

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

^ permalink raw reply	[flat|nested] 14+ messages in thread

* [Buildroot] [Bug 7478] Multiple chosen python modules are not built due to missing header files in include path
  2014-09-27 22:49 [Buildroot] [Bug 7478] New: Multiple chosen python modules are not built due to missing header files in include path bugzilla at busybox.net
@ 2014-09-27 22:50 ` bugzilla at busybox.net
  2014-10-02 19:45 ` bugzilla at busybox.net
                   ` (11 subsequent siblings)
  12 siblings, 0 replies; 14+ messages in thread
From: bugzilla at busybox.net @ 2014-09-27 22:50 UTC (permalink / raw)
  To: buildroot

https://bugs.busybox.net/show_bug.cgi?id=7478

--- Comment #1 from Christian Abegg <cma@vtxnet.ch> 2014-09-27 22:50:28 UTC ---
Created attachment 5690
  --> https://bugs.busybox.net/attachment.cgi?id=5690
.config file used

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

^ permalink raw reply	[flat|nested] 14+ messages in thread

* [Buildroot] [Bug 7478] Multiple chosen python modules are not built due to missing header files in include path
  2014-09-27 22:49 [Buildroot] [Bug 7478] New: Multiple chosen python modules are not built due to missing header files in include path bugzilla at busybox.net
  2014-09-27 22:50 ` [Buildroot] [Bug 7478] " bugzilla at busybox.net
@ 2014-10-02 19:45 ` bugzilla at busybox.net
  2014-10-02 19:54 ` bugzilla at busybox.net
                   ` (10 subsequent siblings)
  12 siblings, 0 replies; 14+ messages in thread
From: bugzilla at busybox.net @ 2014-10-02 19:45 UTC (permalink / raw)
  To: buildroot

https://bugs.busybox.net/show_bug.cgi?id=7478

--- Comment #2 from Thomas De Schampheleire <patrickdepinguin@gmail.com> 2014-10-02 19:45:50 UTC ---
Thanks a lot for your detailed bug report. I will try to reproduce this issue
and report back.

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

^ permalink raw reply	[flat|nested] 14+ messages in thread

* [Buildroot] [Bug 7478] Multiple chosen python modules are not built due to missing header files in include path
  2014-09-27 22:49 [Buildroot] [Bug 7478] New: Multiple chosen python modules are not built due to missing header files in include path bugzilla at busybox.net
  2014-09-27 22:50 ` [Buildroot] [Bug 7478] " bugzilla at busybox.net
  2014-10-02 19:45 ` bugzilla at busybox.net
@ 2014-10-02 19:54 ` bugzilla at busybox.net
  2014-10-05 13:06 ` bugzilla at busybox.net
                   ` (9 subsequent siblings)
  12 siblings, 0 replies; 14+ messages in thread
From: bugzilla at busybox.net @ 2014-10-02 19:54 UTC (permalink / raw)
  To: buildroot

https://bugs.busybox.net/show_bug.cgi?id=7478

--- Comment #3 from Thomas De Schampheleire <patrickdepinguin@gmail.com> 2014-10-02 19:54:20 UTC ---
With this qemu-x86_64 defconfig :
BR2_x86_64=y
BR2_DL_DIR="~/repo/contrib/dl"
BR2_TOOLCHAIN_EXTERNAL=y
BR2_TARGET_GENERIC_GETTY_PORT="tty1"
BR2_LINUX_KERNEL=y
BR2_LINUX_KERNEL_CUSTOM_VERSION=y
BR2_LINUX_KERNEL_CUSTOM_VERSION_VALUE="3.13.5"
BR2_LINUX_KERNEL_USE_CUSTOM_CONFIG=y
BR2_LINUX_KERNEL_CUSTOM_CONFIG_FILE="board/qemu/x86_64/linux-3.13.config"
BR2_PACKAGE_PYTHON=y
BR2_PACKAGE_PYTHON_SQLITE=y
BR2_TARGET_ROOTFS_EXT2=y
# BR2_TARGET_ROOTFS_TAR is not set

I see similar messages during the build:

Python build finished, but the necessary bits to build these modules were not
found:
_bsddb             _curses            _curses_panel   
_ssl               _tkinter           bsddb185        
bz2                dbm                dl              
gdbm               imageop            linuxaudiodev   
ossaudiodev        readline           sunaudiodev     
zlib

With this build (which is a slightly different config as the submitter showed)
sqlite3 can be imported correctly, but ssl cannot (_ssl not found, same thing
as reported).

So indeed, something seems incorrect here. 
I will try to investigate further.

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

^ permalink raw reply	[flat|nested] 14+ messages in thread

* [Buildroot] [Bug 7478] Multiple chosen python modules are not built due to missing header files in include path
  2014-09-27 22:49 [Buildroot] [Bug 7478] New: Multiple chosen python modules are not built due to missing header files in include path bugzilla at busybox.net
                   ` (2 preceding siblings ...)
  2014-10-02 19:54 ` bugzilla at busybox.net
@ 2014-10-05 13:06 ` bugzilla at busybox.net
  2014-10-12 20:38 ` bugzilla at busybox.net
                   ` (8 subsequent siblings)
  12 siblings, 0 replies; 14+ messages in thread
From: bugzilla at busybox.net @ 2014-10-05 13:06 UTC (permalink / raw)
  To: buildroot

https://bugs.busybox.net/show_bug.cgi?id=7478

--- Comment #4 from Thomas De Schampheleire <patrickdepinguin@gmail.com> 2014-10-05 13:06:05 UTC ---
I did some further investigation: 

a. the build message "Python build finished, but the necessary bits to build
these modules were not found" occurs when a python module requires some
external support but this support is not present.
For example, when SSL is not selected in the buildroot configuration, then you
get this message for the 'ssl' python module. In this case '_ssl' will be in
the list.

b. if I build your configuration (but use an external toolchain and disable the
kernel, htop, vlc, util-linux, and grub, to save build time), I do not see the
same issue. I do get a message about some bits not found, but not for ssl,
sqlite, nor any of the other modules you selected.
This makes sense: when these python modules are selected in the python
configuration, the corresponding dependencies like the ssl package, the sqlite
package, etc. are built automatically.

This makes me believe that there may be a one-time issue due to some particular
action you did on your buildroot directory.
Have you tried starting from a clean build, i.e. run
'make clean all'

Does the problem occur again?

Thanks,
Thomas

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

^ permalink raw reply	[flat|nested] 14+ messages in thread

* [Buildroot] [Bug 7478] Multiple chosen python modules are not built due to missing header files in include path
  2014-09-27 22:49 [Buildroot] [Bug 7478] New: Multiple chosen python modules are not built due to missing header files in include path bugzilla at busybox.net
                   ` (3 preceding siblings ...)
  2014-10-05 13:06 ` bugzilla at busybox.net
@ 2014-10-12 20:38 ` bugzilla at busybox.net
  2014-10-13  9:35 ` bugzilla at busybox.net
                   ` (7 subsequent siblings)
  12 siblings, 0 replies; 14+ messages in thread
From: bugzilla at busybox.net @ 2014-10-12 20:38 UTC (permalink / raw)
  To: buildroot

https://bugs.busybox.net/show_bug.cgi?id=7478

--- Comment #5 from Christian Abegg <cma@vtxnet.ch> 2014-10-12 20:38:21 UTC ---
Thanks a lot for your reply.

a. This seems logical. However, chosen modules in the configuration where
showing up in that list.

b. I retried multiple builds with the latest version of buildroot (git
526e3d0). What I observed is that when using the buildroot internal toolchain,
the python sqlite3 modules were never built. On the other hand, when using an
external toolchain (I used Sourcery CodeBench x86/x86_64 2012.09) the python
sqlite3 modules were correctly built.

So using an external toolchain solves the problem, which is a bit strange.

Here are some excerpts of the python build log, 

using the buildroot toolchain:
building '_curses' extension
/tmp/buildroot/output/host/usr/bin/i686-buildroot-linux-gnu-gcc -fPIC
-fno-strict-aliasing -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE
-D_FILE_OFFSET_BITS=64 -pipe -Os -DNDEBUG -g -fwrapv -O3 -Wall
-Wstrict-prototypes -I. -IInclude -I./Include
-I/tmp/buildroot/output/build/python-2.7.8/Include
-I/tmp/buildroot/output/build/python-2.7.8 -c
/tmp/buildroot/output/build/python-2.7.8/Modules/_cursesmodule.c -o
build/temp.linux2-i686-2.7/tmp/buildroot/output/build/python-2.7.8/Modules/_cursesmodule.o

using an external toolchain:
building '_curses' extension
/tmp/buildroot/output/host/usr/bin/i686-pc-linux-gnu-gcc -fPIC
-fno-strict-aliasing -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE
-D_FILE_OFFSET_BITS=64 -pipe -Os -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I.
-IInclude -I./Include
-I/tmp/buildroot/output/host/usr/i686-buildroot-linux-gnu/sysroot/usr/include
-I/tmp/buildroot/output/build/python-2.7.8/Include
-I/tmp/buildroot/output/build/python-2.7.8 -c
/tmp/buildroot/output/build/python-2.7.8/Modules/_cursesmodule.c -o
build/temp.linux2-i686-2.7/tmp/buildroot/output/build/python-2.7.8/Modules/_cursesmodule.o

The difference I see between the internal and external toolchain, is that with
the external one python is built with an additional include path:
-I/tmp/buildroot/output/host/usr/i686-buildroot-linux-gnu/sysroot/usr/include.
In this directory are the sqlite3.h headers among others.

You said you used an external toolchain and the python sqlite module built
fine. If you use the internal toolchain, does the module also build correctly?

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

^ permalink raw reply	[flat|nested] 14+ messages in thread

* [Buildroot] [Bug 7478] Multiple chosen python modules are not built due to missing header files in include path
  2014-09-27 22:49 [Buildroot] [Bug 7478] New: Multiple chosen python modules are not built due to missing header files in include path bugzilla at busybox.net
                   ` (4 preceding siblings ...)
  2014-10-12 20:38 ` bugzilla at busybox.net
@ 2014-10-13  9:35 ` bugzilla at busybox.net
  2014-10-13 14:30 ` bugzilla at busybox.net
                   ` (6 subsequent siblings)
  12 siblings, 0 replies; 14+ messages in thread
From: bugzilla at busybox.net @ 2014-10-13  9:35 UTC (permalink / raw)
  To: buildroot

https://bugs.busybox.net/show_bug.cgi?id=7478

--- Comment #6 from Thomas De Schampheleire <patrickdepinguin@gmail.com> 2014-10-13 09:35:02 UTC ---
I now tried using an internal toolchain:
BR2_x86_64=y
BR2_DL_DIR="~/repo/contrib/dl"
BR2_TOOLCHAIN_BUILDROOT_WCHAR=y
BR2_TARGET_GENERIC_GETTY_PORT="tty1"
BR2_LINUX_KERNEL=y
BR2_LINUX_KERNEL_CUSTOM_VERSION=y
BR2_LINUX_KERNEL_CUSTOM_VERSION_VALUE="3.13.5"
BR2_LINUX_KERNEL_USE_CUSTOM_CONFIG=y
BR2_LINUX_KERNEL_CUSTOM_CONFIG_FILE="board/qemu/x86_64/linux-3.13.config"
BR2_PACKAGE_PYTHON=y
BR2_PACKAGE_PYTHON_SQLITE=y
BR2_TARGET_ROOTFS_EXT2=y
# BR2_TARGET_ROOTFS_TAR is not set

but this seems to build fine too, wrt sqlite.

In the build log I also see the mentioned include paths present, for example:

building '_sqlite3' extension
creating
build/temp.linux2-x86_64-2.7/home/tdescham/repo/contrib/buildroot-reproduce/output/build/python-2.7.8/Modules/_sqlite
/home/tdescham/repo/contrib/buildroot-reproduce/output/host/usr/bin/x86_64-buildroot-linux-uclibc-gcc
-fPIC -fno-strict-aliasing -pipe -Os -DNDEBUG -g -fwrapv -O3 -Wall
-Wstrict-prototypes -DMODULE_NAME="sqlite3" -DSQLITE_OMIT_LOAD_EXTENSION=1
-IModules/_sqlite
-I/home/tdescham/repo/contrib/buildroot-reproduce/output/host/usr/x86_64-buildroot-linux-uclibc/sysroot/usr/include
-I. -IInclude -I./Include
-I/home/tdescham/repo/contrib/buildroot-reproduce/output/host/usr/x86_64-buildroot-linux-uclibc/sysroot/usr/include
-I/home/tdescham/repo/contrib/buildroot-reproduce/output/build/python-2.7.8/Include
-I/home/tdescham/repo/contrib/buildroot-reproduce/output/build/python-2.7.8 -c
/home/tdescham/repo/contrib/buildroot-reproduce/output/build/python-2.7.8/Modules/_sqlite/cache.c
-o
build/temp.linux2-x86_64-2.7/home/tdescham/repo/contrib/buildroot-reproduce/output/build/python-2.7.8/Modules/_sqlite/cache.o


Your defconfig was:
BR2_x86_i686=y
BR2_TOOLCHAIN_BUILDROOT_GLIBC=y
BR2_TOOLCHAIN_BUILDROOT_CXX=y
BR2_TARGET_GENERIC_GETTY_PORT="tty1"
BR2_LINUX_KERNEL=y
BR2_LINUX_KERNEL_DEFCONFIG="i386"
BR2_LINUX_KERNEL_INSTALL_TARGET=y
BR2_PACKAGE_VLC=y
BR2_PACKAGE_PYTHON=y
BR2_PACKAGE_PYTHON_PY_PYC=y
BR2_PACKAGE_PYTHON_BZIP2=y
BR2_PACKAGE_PYTHON_BSDDB=y
BR2_PACKAGE_PYTHON_CODECSCJK=y
BR2_PACKAGE_PYTHON_CURSES=y
BR2_PACKAGE_PYTHON_PYEXPAT=y
BR2_PACKAGE_PYTHON_READLINE=y
BR2_PACKAGE_PYTHON_SSL=y
BR2_PACKAGE_PYTHON_SQLITE=y
BR2_PACKAGE_PYTHON_ZLIB=y
BR2_PACKAGE_PYTHON_HASHLIB=y
BR2_PACKAGE_OPENSSH=y
BR2_PACKAGE_HTOP=y
BR2_PACKAGE_UTIL_LINUX=y
BR2_TARGET_ROOTFS_CPIO=y
BR2_TARGET_ROOTFS_EXT2=y
BR2_TARGET_GRUB=y
BR2_TARGET_GRUB_FS_ISO9660=y

so the main differences I see are:
- 32-bit
- glibc
- c++ support

I will now make a build with glibc and c++ to see if it makes a difference...

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

^ permalink raw reply	[flat|nested] 14+ messages in thread

* [Buildroot] [Bug 7478] Multiple chosen python modules are not built due to missing header files in include path
  2014-09-27 22:49 [Buildroot] [Bug 7478] New: Multiple chosen python modules are not built due to missing header files in include path bugzilla at busybox.net
                   ` (5 preceding siblings ...)
  2014-10-13  9:35 ` bugzilla at busybox.net
@ 2014-10-13 14:30 ` bugzilla at busybox.net
  2014-10-20 19:05 ` bugzilla at busybox.net
                   ` (5 subsequent siblings)
  12 siblings, 0 replies; 14+ messages in thread
From: bugzilla at busybox.net @ 2014-10-13 14:30 UTC (permalink / raw)
  To: buildroot

https://bugs.busybox.net/show_bug.cgi?id=7478

--- Comment #7 from Thomas De Schampheleire <patrickdepinguin@gmail.com> 2014-10-13 14:30:50 UTC ---
Even enabling C++ and glibc works fine here:

BR2_x86_64=y
BR2_DL_DIR="~/repo/contrib/dl"
BR2_TOOLCHAIN_BUILDROOT_GLIBC=y
BR2_TOOLCHAIN_BUILDROOT_CXX=y
BR2_TARGET_GENERIC_GETTY_PORT="tty1"
BR2_LINUX_KERNEL=y
BR2_LINUX_KERNEL_CUSTOM_VERSION=y
BR2_LINUX_KERNEL_CUSTOM_VERSION_VALUE="3.13.5"
BR2_LINUX_KERNEL_USE_CUSTOM_CONFIG=y
BR2_LINUX_KERNEL_CUSTOM_CONFIG_FILE="board/qemu/x86_64/linux-3.13.config"
BR2_PACKAGE_PYTHON=y
BR2_PACKAGE_PYTHON_SQLITE=y
BR2_TARGET_ROOTFS_EXT2=y
# BR2_TARGET_ROOTFS_TAR is not set


Build log snippet:
building '_sqlite3' extension
creating
build/temp.linux2-x86_64-2.7/home/tdescham/repo/contrib/buildroot-reproduce/output/build/python-2.7.8/Modules/_sqlite
/home/tdescham/repo/contrib/buildroot-reproduce/output/host/usr/bin/x86_64-buildroot-linux-gnu-gcc
-fPIC -fno-strict-aliasing -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE
-D_FILE_OFFSET_BITS=64 -pipe -Os -DNDEBUG -g -fwrapv -O3 -Wall
-Wstrict-prototypes -DMODULE_NAME="sqlite3" -DSQLITE_OMIT_LOAD_EXTENSION=1
-IModules/_sqlite
-I/home/tdescham/repo/contrib/buildroot-reproduce/output/host/usr/x86_64-buildroot-linux-gnu/sysroot/usr/include
-I. -IInclude -I./Include
-I/home/tdescham/repo/contrib/buildroot-reproduce/output/host/usr/x86_64-buildroot-linux-gnu/sysroot/usr/include
-I/home/tdescham/repo/contrib/buildroot-reproduce/output/build/python-2.7.8/Include
-I/home/tdescham/repo/contrib/buildroot-reproduce/output/build/python-2.7.8 -c
/home/tdescham/repo/contrib/buildroot-reproduce/output/build/python-2.7.8/Modules/_sqlite/cache.c
-o
build/temp.linux2-x86_64-2.7/home/tdescham/repo/contrib/buildroot-reproduce/output/build/python-2.7.8/Modules/_sqlite/cache.o


Could you try any of these working configurations (build-wise) to check if you
see something differently? I'm testing this on a recent git revision, I haven't
verified if there were changes in this area.


In the Python sources, Lib/distutils/command/build_ext.py seems to be
responsible for building extensions, see function build_extension. I think it
makes sense to debug this further, inspecting the values of ext.include_dirs.

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

^ permalink raw reply	[flat|nested] 14+ messages in thread

* [Buildroot] [Bug 7478] Multiple chosen python modules are not built due to missing header files in include path
  2014-09-27 22:49 [Buildroot] [Bug 7478] New: Multiple chosen python modules are not built due to missing header files in include path bugzilla at busybox.net
                   ` (6 preceding siblings ...)
  2014-10-13 14:30 ` bugzilla at busybox.net
@ 2014-10-20 19:05 ` bugzilla at busybox.net
  2014-10-20 22:43 ` bugzilla at busybox.net
                   ` (4 subsequent siblings)
  12 siblings, 0 replies; 14+ messages in thread
From: bugzilla at busybox.net @ 2014-10-20 19:05 UTC (permalink / raw)
  To: buildroot

https://bugs.busybox.net/show_bug.cgi?id=7478

--- Comment #8 from Thomas De Schampheleire <patrickdepinguin@gmail.com> 2014-10-20 19:05:18 UTC ---
Christian, do you have further feedback on this problem?
I cannot reproduce the failure here, but since you detected a specific
difference, being the different include paths, I think it would be good to
debug at your end by adding some debug statements in the Python file I
mentioned in comment #7.

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

^ permalink raw reply	[flat|nested] 14+ messages in thread

* [Buildroot] [Bug 7478] Multiple chosen python modules are not built due to missing header files in include path
  2014-09-27 22:49 [Buildroot] [Bug 7478] New: Multiple chosen python modules are not built due to missing header files in include path bugzilla at busybox.net
                   ` (7 preceding siblings ...)
  2014-10-20 19:05 ` bugzilla at busybox.net
@ 2014-10-20 22:43 ` bugzilla at busybox.net
  2014-10-20 22:44 ` bugzilla at busybox.net
                   ` (3 subsequent siblings)
  12 siblings, 0 replies; 14+ messages in thread
From: bugzilla at busybox.net @ 2014-10-20 22:43 UTC (permalink / raw)
  To: buildroot

https://bugs.busybox.net/show_bug.cgi?id=7478

--- Comment #9 from Christian Abegg <cma@vtxnet.ch> 2014-10-20 22:43:18 UTC ---
Hi Thomas,

thanks for the hints and sorry for the delay.

I adapted my config file with your options and still got the same errors. I
also tried glibc and uclib and gcc4.8/4.9 but the sqlite extension is still not
built. As soon as I switch to an external toolchain, it works correctly.

For the include paths, I added some print statements in the
Lib/distutils/command/build_ext.py and the setup.py file (see
br-python-2.7.8-setup-py.patch attachment). As seen below, the sysroot include
dir isn't received by the python setup script (setup.py):

# With buildroot internal toolchain
inc_dirs =  ['.', 'Include', './Include',
'/tmp/buildroot-internal/output/build/python-2.7.8/Include',
'/tmp/buildroot-internal/output/build/python-2.7.8']

# Using an external toolchain
inc_dirs =  ['.', 'Include', './Include',
'/tmp/buildroot-external/output/host/usr/i686-buildroot-linux-gnu/sysroot/usr/include',
'/tmp/buildroot-external/output/build/python-2.7.8/Include',
'/tmp/buildroot-external/output/build/python-2.7.8']
and from the build_ext.py file:
ext.include_dirs = ['Modules/_sqlite',
'/tmp/buildroot-external/output/host/usr/i686-buildroot-linux-gnu/sysroot/usr/include']

So somehow the required include dir
('/tmp/buildroot-external/output/host/usr/i686-buildroot-linux-gnu/sysroot/usr/include')
isn't forwarded to the python setup.py script who builds the extensions.

From that point, I'm a bit clueless why the sysroot include dir isn't in the
dir list when an internal toolchain is used. Where is the next place to look ?
Maybe something with the python configure options ?

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

^ permalink raw reply	[flat|nested] 14+ messages in thread

* [Buildroot] [Bug 7478] Multiple chosen python modules are not built due to missing header files in include path
  2014-09-27 22:49 [Buildroot] [Bug 7478] New: Multiple chosen python modules are not built due to missing header files in include path bugzilla at busybox.net
                   ` (8 preceding siblings ...)
  2014-10-20 22:43 ` bugzilla at busybox.net
@ 2014-10-20 22:44 ` bugzilla at busybox.net
  2014-10-26 21:01 ` bugzilla at busybox.net
                   ` (2 subsequent siblings)
  12 siblings, 0 replies; 14+ messages in thread
From: bugzilla at busybox.net @ 2014-10-20 22:44 UTC (permalink / raw)
  To: buildroot

https://bugs.busybox.net/show_bug.cgi?id=7478

--- Comment #10 from Christian Abegg <cma@vtxnet.ch> 2014-10-20 22:44:04 UTC ---
Created attachment 5738
  --> https://bugs.busybox.net/attachment.cgi?id=5738
patch setup.py to print include dir list

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

^ permalink raw reply	[flat|nested] 14+ messages in thread

* [Buildroot] [Bug 7478] Multiple chosen python modules are not built due to missing header files in include path
  2014-09-27 22:49 [Buildroot] [Bug 7478] New: Multiple chosen python modules are not built due to missing header files in include path bugzilla at busybox.net
                   ` (9 preceding siblings ...)
  2014-10-20 22:44 ` bugzilla at busybox.net
@ 2014-10-26 21:01 ` bugzilla at busybox.net
  2014-10-29 23:30 ` bugzilla at busybox.net
  2015-04-10 15:18 ` bugzilla at busybox.net
  12 siblings, 0 replies; 14+ messages in thread
From: bugzilla at busybox.net @ 2014-10-26 21:01 UTC (permalink / raw)
  To: buildroot

https://bugs.busybox.net/show_bug.cgi?id=7478

--- Comment #11 from Thomas Petazzoni <thomas.petazzoni@free-electrons.com> 2014-10-26 21:01:14 UTC ---
(In reply to comment #9)
> From that point, I'm a bit clueless why the sysroot include dir isn't in the
> dir list when an internal toolchain is used. Where is the next place to look ?
> Maybe something with the python configure options ?

There is no need for the sysroot include dir to be explicitly passed to gcc
with the internal toolchain, because the Buildroot sysroot is configured to be
the natural location for the compiler to look for headers.

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

^ permalink raw reply	[flat|nested] 14+ messages in thread

* [Buildroot] [Bug 7478] Multiple chosen python modules are not built due to missing header files in include path
  2014-09-27 22:49 [Buildroot] [Bug 7478] New: Multiple chosen python modules are not built due to missing header files in include path bugzilla at busybox.net
                   ` (10 preceding siblings ...)
  2014-10-26 21:01 ` bugzilla at busybox.net
@ 2014-10-29 23:30 ` bugzilla at busybox.net
  2015-04-10 15:18 ` bugzilla at busybox.net
  12 siblings, 0 replies; 14+ messages in thread
From: bugzilla at busybox.net @ 2014-10-29 23:30 UTC (permalink / raw)
  To: buildroot

https://bugs.busybox.net/show_bug.cgi?id=7478

--- Comment #12 from Christian Abegg <cma@vtxnet.ch> 2014-10-29 23:30:26 UTC ---
(In reply to comment #11)
> There is no need for the sysroot include dir to be explicitly passed to gcc
> with the internal toolchain, because the Buildroot sysroot is configured to be
> the natural location for the compiler to look for headers.

Ok, thanks for the details. However I'm still wondering why the inc path array
in the setup.py script doesn't contain the sysroot include dir when the
internal toolchain. What is the next place were I can check why this isn't the
case?

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

^ permalink raw reply	[flat|nested] 14+ messages in thread

* [Buildroot] [Bug 7478] Multiple chosen python modules are not built due to missing header files in include path
  2014-09-27 22:49 [Buildroot] [Bug 7478] New: Multiple chosen python modules are not built due to missing header files in include path bugzilla at busybox.net
                   ` (11 preceding siblings ...)
  2014-10-29 23:30 ` bugzilla at busybox.net
@ 2015-04-10 15:18 ` bugzilla at busybox.net
  12 siblings, 0 replies; 14+ messages in thread
From: bugzilla at busybox.net @ 2015-04-10 15:18 UTC (permalink / raw)
  To: buildroot

https://bugs.busybox.net/show_bug.cgi?id=7478

Thomas Petazzoni <thomas.petazzoni@free-electrons.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|                            |WORKSFORME

--- Comment #13 from Thomas Petazzoni <thomas.petazzoni@free-electrons.com> 2015-04-10 15:18:22 UTC ---
I tried again today to do a build with an internal toolchain, python and sqlite
enabled, and it works fine, and I can import the sqlite3 python module on the
target successfully:

Welcome to Buildroot
buildroot login: root
# python -c "import sqlite3"
# 

So, I'm marking this bug as "works for me", since apparently neither Thomas DS
nor I have been able to reproduce it so far.

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

^ permalink raw reply	[flat|nested] 14+ messages in thread

end of thread, other threads:[~2015-04-10 15:18 UTC | newest]

Thread overview: 14+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2014-09-27 22:49 [Buildroot] [Bug 7478] New: Multiple chosen python modules are not built due to missing header files in include path bugzilla at busybox.net
2014-09-27 22:50 ` [Buildroot] [Bug 7478] " bugzilla at busybox.net
2014-10-02 19:45 ` bugzilla at busybox.net
2014-10-02 19:54 ` bugzilla at busybox.net
2014-10-05 13:06 ` bugzilla at busybox.net
2014-10-12 20:38 ` bugzilla at busybox.net
2014-10-13  9:35 ` bugzilla at busybox.net
2014-10-13 14:30 ` bugzilla at busybox.net
2014-10-20 19:05 ` bugzilla at busybox.net
2014-10-20 22:43 ` bugzilla at busybox.net
2014-10-20 22:44 ` bugzilla at busybox.net
2014-10-26 21:01 ` bugzilla at busybox.net
2014-10-29 23:30 ` bugzilla at busybox.net
2015-04-10 15:18 ` bugzilla at busybox.net

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.