[Buildroot] [Bug 7268] New: python 2.7 compilation issue

[Buildroot] [Bug 7268] New: python 2.7 compilation issue

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=7268

           Summary: python 2.7 compilation issue
           Product: buildroot
           Version: unspecified
          Platform: PC
        OS/Version: Linux
            Status: NEW
          Severity: normal
          Priority: P5
         Component: Other
        AssignedTo: unassigned at buildroot.uclibc.org
        ReportedBy: buildroot-2014 at inbox.com
                CC: buildroot at uclibc.org
   Estimated Hours: 0.0


Hi,

I am trying to build python for an x86_64 target. 

I am using fresh sources of buildroot (exactly the git revision
5ac8f8ce10027ac8285a7e9cec16e74aa38ee2c1).

To achieve that, I go through following steps :
- make defconfig
- in Target Options, change Target Architecture to x86_64 / generic
- in Toolchain, change C Library to glibc
- in Target Packages / Interpreter languages and scripting, enable python
- make

The following tarball encloses configuration file and make output.

It seems host part of python build goes well. Target part, however, fails
building "struct" module (I'm not a python expert... so I may be wrong).

Thanks

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Buildroot] [Bug 7268] python 2.7 compilation issue

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=7268

--- Comment #1 from David <buildroot-2014@inbox.com> 2014-07-07 17:18:26 UTC ---
Created attachment 5486
  --> https://bugs.busybox.net/attachment.cgi?id=5486
buildroot configuration

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Buildroot] [Bug 7268] python 2.7 compilation issue

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=7268

--- Comment #2 from David <buildroot-2014@inbox.com> 2014-07-07 17:24:42 UTC ---
Created attachment 5492
  --> https://bugs.busybox.net/attachment.cgi?id=5492
Make output

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Buildroot] [Bug 7268] python 2.7 compilation issue

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=7268

--- Comment #3 from David <buildroot-2014@inbox.com> 2014-07-07 17:26:24 UTC ---
Hi,

if needed, here is the full make output (5MB, but 797MB uncompressed) :
http://dl.free.fr/eMphhZaBN

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Buildroot] [Bug 7268] python 2.7 compilation issue

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=7268

--- Comment #4 from Thomas Petazzoni <thomas.petazzoni@free-electrons.com> 2014-07-07 17:32:16 UTC ---
Thanks for the report, I've just started a build with your configuration to try
to reproduce the problem.

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Buildroot] [Bug 7268] python 2.7 compilation issue

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=7268

--- Comment #5 from Thomas Petazzoni <thomas.petazzoni@free-electrons.com> 2014-07-07 17:47:44 UTC ---
The build of your .config works fine here (Debian GNU/Linux 6.0).

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Buildroot] [Bug 7268] python 2.7 compilation issue

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=7268

--- Comment #6 from Thomas Petazzoni <thomas.petazzoni@free-electrons.com> 2014-07-07 19:18:57 UTC ---
Ok, so the problem is due to the fact that for some reason in your case, Python
has added "-I/usr/include/x86_64-linux-gnu" to the CFLAGS used when building
target modules, which doesn't make sense. I don't have this flag added when
building here, which explains why it works.

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Buildroot] [Bug 7268] python 2.7 compilation issue

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=7268

--- Comment #7 from Thomas Petazzoni <thomas.petazzoni@free-electrons.com> 2014-07-07 19:20:25 UTC ---
(In reply to comment #3)
> Hi,
> 
> if needed, here is the full make output (5MB, but 797MB uncompressed) :
> http://dl.free.fr/eMphhZaBN

Unfortunately, I'm not able to download it. Could you put it somewhere else?

Thanks.

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Buildroot] [Bug 7268] python 2.7 compilation issue

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=7268

--- Comment #8 from David <buildroot-2014@inbox.com> 2014-07-07 20:38:25 UTC ---
Thanks for your reply

Here is the download link :
http://we.tl/vSYk0yUI5r

I forgot to say : my host is a Debian 7.5

I'll try to remove this flag tomorrow morning

Regards

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Buildroot] [Bug 7268] python 2.7 compilation issue

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=7268

--- Comment #9 from David <buildroot-2014@inbox.com> 2014-07-08 11:51:23 UTC ---
Hi,

A few more details about this issue...

A git bisect identifies 7e960dc9 as the first "bad" commit : I manage to
compile python using revision 577e52ac, but I do not using revision 7e960dc9.

7e960dc9da56d4a484b5480746aaf617ca491274 is the first bad commit
commit 7e960dc9da56d4a484b5480746aaf617ca491274
Author: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Date:   Thu Feb 13 21:48:53 2014 +0100

    python: bump to 2.7.6

    Even though jumping from 2.7.3 to 2.7.6 looks like a minor version
    bump, it is in fact a fairly significant one, because a good number of
    changes to help cross-compilation have been merged into Python
    upstream. Therefore, most of our patches are affected by this change.

    In detail, this commit:

     * Renames all the patches to follow the naming convention of patches
       in Buildroot: the patch file names should not have any version
       number.

     * The patches numbered above 100, that add configuration options to
       disable certain modules of the Python standard library, are only
       renamed and slightly adapted, they didn't change that much.

     * The patches numbered below 100 are almost entirely rewritten: many
       of the cross-compilation problems that used to exist in Python
       2.7.3 no longer exist, and the number of remaining problems is
       smaller, and can be fixed with smaller patches.

    Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
    Signed-off-by: Peter Korsgaard <peter@korsgaard.com>

:040000 040000 8618414254d7998096c5c87b7774cbd50217cb08
ddcec6a08dbe50e52973f7659789200593fe6861 M    package

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Buildroot] [Bug 7268] python 2.7 compilation issue on a Debian/Ubuntu host with multiarch support

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=7268

David <buildroot-2014@inbox.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Summary|python 2.7 compilation      |python 2.7 compilation
                   |issue                       |issue on a Debian/Ubuntu
                   |                            |host with multiarch support

--- Comment #10 from David <buildroot-2014@inbox.com> 2014-07-08 17:04:14 UTC ---
Hi,

I finally got the bug. Your input about wrong paths really helped me and saved
me a lot of time, thanks a lot for this.

In fact, the main setup.py script has a "add_multiarch_paths" function that
adapts standard paths to multiarch requirements. For some reason, it adds host
/usr/include/x86_64-linux-gnu to include paths for the target python build. 

I add a patch that makes multiarch management process available only if python
is not being cross-compiled. I do not know if multiarch really makes sense in
the case of buildroot. This patch fixes my compilation issue, but it may break
something for a different architecture, so I hope someone with a lot more
experience about python and buildroot architectures than me will read this
patch.

Thanks a lot again
David

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Buildroot] [Bug 7268] python 2.7 compilation issue on a Debian/Ubuntu host with multiarch support

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=7268

--- Comment #11 from David <buildroot-2014@inbox.com> 2014-07-08 17:05:20 UTC ---
Created attachment 5504
  --> https://bugs.busybox.net/attachment.cgi?id=5504
manage multiarch paths only if not cross-compiling

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Buildroot] [Bug 7268] python 2.7 compilation issue on a Debian/Ubuntu host with multiarch support

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=7268

Yann E. MORIN <yann.morin.1998@free.fr> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |yann.morin.1998 at free.fr

--- Comment #12 from Yann E. MORIN <yann.morin.1998@free.fr> 2014-07-08 17:19:34 UTC ---
David,

Your patch is reversed. ;-)

Care to send the patch to the list, please?

Thanks.

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Buildroot] [Bug 7268] python 2.7 compilation issue on a Debian/Ubuntu host with multiarch support

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=7268

--- Comment #13 from Thomas Petazzoni <thomas.petazzoni@free-electrons.com> 2014-07-08 20:20:50 UTC ---
Patch for python posted at http://patchwork.ozlabs.org/patch/368000/, and patch
for python3 posted at http://patchwork.ozlabs.org/patch/368001/.

Thanks a lot David for your bug report and investigation! Once the patches are
merged by Peter, we'll mark this bug as fixed. Of course, it would be good if
you could formally test the patches, and give your Tested-by on the patches if
their work for you.

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Buildroot] [Bug 7268] python 2.7 compilation issue on a Debian/Ubuntu host with multiarch support

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=7268

--- Comment #14 from David Bachelart <david.bachelart@bbright.com> 2014-07-09 08:56:48 UTC ---
Hi Thomas,

I managed to compile python 2.7 and python 3 using your patches.

Please forgive me, I'm not familiar with the "Tested-by" thing... How would you
do that ?

Thanks
David

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Buildroot] [Bug 7268] python 2.7 compilation issue on a Debian/Ubuntu host with multiarch support

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=7268

--- Comment #15 from Thomas Petazzoni <thomas.petazzoni@free-electrons.com> 2014-07-09 11:26:37 UTC ---
(In reply to comment #14)
> Hi Thomas,
> 
> I managed to compile python 2.7 and python 3 using your patches.
> 
> Please forgive me, I'm not familiar with the "Tested-by" thing... How would you
> do that ?

You would have had to reply to the patches sent to the mailing list, and just
add a line like:

Tested-by: John Doe <john.doe@foobar.org>

In the reply.

However, the patches have been committed now, so it's too late :-)

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Buildroot] [Bug 7268] python 2.7 compilation issue on a Debian/Ubuntu host with multiarch support

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=7268

Thomas Petazzoni <thomas.petazzoni@free-electrons.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|                            |FIXED

--- Comment #16 from Thomas Petazzoni <thomas.petazzoni@free-electrons.com> 2014-07-09 11:27:10 UTC ---
Python 2.x fixed in
http://git.buildroot.net/buildroot/commit/?id=5476b7273f0e5e58b59da9bf6403366d08e6b585,
and Python 3.x fixed in
http://git.buildroot.net/buildroot/commit/?id=5364b22b6c309073d59f593c3b5b1a9b3246e957.

Closing bug.

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.