[Buildroot] [PATCH] package/libftdi1: fix python support

[Buildroot] [PATCH] package/libftdi1: fix python support

[yegorslists at googlemail.com]

From: Yegor Yefremov <yegorslists@googlemail.com>

Add an upstreamed patch that reorders find_package() commands.
This way Python interpreter will be detected first and based on
it the Python libraries can be found.

Fixes the following CMake error:

Could NOT find PythonLibs (missing: PYTHON_LIBRARIES PYTHON_INCLUDE_DIRS)

Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
---
 .../0004-CMake-reorder-Python-detection.patch | 32 +++++++++++++++++++
 1 file changed, 32 insertions(+)
 create mode 100644 package/libftdi1/0004-CMake-reorder-Python-detection.patch

diff --git a/package/libftdi1/0004-CMake-reorder-Python-detection.patch b/package/libftdi1/0004-CMake-reorder-Python-detection.patch
new file mode 100644
index 0000000000..b0eedf07d4
--- /dev/null
+++ b/package/libftdi1/0004-CMake-reorder-Python-detection.patch
@@ -0,0 +1,32 @@
+From 640c2b3c0c72d5194d422ff7dda086c4533fba00 Mon Sep 17 00:00:00 2001
+From: Yegor Yefremov <yegorslists@googlemail.com>
+Date: Mon, 13 Jan 2020 14:08:44 +0100
+Subject: [PATCH 4/4] CMake: reorder Python detection
+
+PythonInterp should be detected before PythonLibs otherwise
+CMake exits with the following message:
+
+Could NOT find PythonLibs (missing: PYTHON_LIBRARIES PYTHON_INCLUDE_DIRS)
+
+Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
+---
+ python/CMakeLists.txt | 2 +-
+ 1 file changed, 1 insertion(+), 1 deletion(-)
+
+diff --git a/python/CMakeLists.txt b/python/CMakeLists.txt
+index 90811ca..368bb7e 100644
+--- a/python/CMakeLists.txt
++++ b/python/CMakeLists.txt
+@@ -12,8 +12,8 @@ if ( PYTHON_BINDINGS )
+       set ( SWIG_FOUND TRUE )
+     endif ()
+   endif ()
+-  find_package ( PythonLibs )
+   find_package ( PythonInterp )
++  find_package ( PythonLibs )
+ endif ()
+ 
+ if ( SWIG_FOUND AND PYTHONLIBS_FOUND AND PYTHONINTERP_FOUND )
+-- 
+2.17.0
+
-- 
2.17.0

[Buildroot] [PATCH] package/libftdi1: fix python support

[Peter Korsgaard]

>>>>> "yegorslists" == yegorslists  <yegorslists@googlemail.com> writes:

 > From: Yegor Yefremov <yegorslists@googlemail.com>
 > Add an upstreamed patch that reorders find_package() commands.
 > This way Python interpreter will be detected first and based on
 > it the Python libraries can be found.

 > Fixes the following CMake error:

 > Could NOT find PythonLibs (missing: PYTHON_LIBRARIES PYTHON_INCLUDE_DIRS)

We don't seem to have any autobuilder failures in libftdi1 for almost 1
year. Any idea why this doesn't trigger on the autobuilders?

-- 
Bye, Peter Korsgaard

[Buildroot] [PATCH] package/libftdi1: fix python support

[Yegor Yefremov]

Hi Peter,

On Fri, Feb 7, 2020 at 10:13 AM Peter Korsgaard <peter@korsgaard.com> wrote:
>
> >>>>> "yegorslists" == yegorslists  <yegorslists@googlemail.com> writes:
>
>  > From: Yegor Yefremov <yegorslists@googlemail.com>
>  > Add an upstreamed patch that reorders find_package() commands.
>  > This way Python interpreter will be detected first and based on
>  > it the Python libraries can be found.
>
>  > Fixes the following CMake error:
>
>  > Could NOT find PythonLibs (missing: PYTHON_LIBRARIES PYTHON_INCLUDE_DIRS)
>
> We don't seem to have any autobuilder failures in libftdi1 for almost 1
> year. Any idea why this doesn't trigger on the autobuilders?

In the version 1.4 find_package() calls were made without the REQUIRED
keyword. Hence, this error wasn't detected by the autobuilders. This
is now fixed upstream and I hope a new release will appear soon. But
meanwhile let's apply this fix.
Alternatively I could bump libftdi1 via a git instead of getting a
source tar file.

I'm preparing a libftdi1 test for Python for BR's CI. So that we could
catch such errors in the future.

Yegor

[Buildroot] [PATCH] package/libftdi1: fix python support

[Peter Korsgaard]

>>>>> "Yegor" == Yegor Yefremov <yegorslists@googlemail.com> writes:

 > Hi Peter,
 > On Fri, Feb 7, 2020 at 10:13 AM Peter Korsgaard <peter@korsgaard.com> wrote:
 >> 
 >> >>>>> "yegorslists" == yegorslists  <yegorslists@googlemail.com> writes:
 >> 
 >> > From: Yegor Yefremov <yegorslists@googlemail.com>
 >> > Add an upstreamed patch that reorders find_package() commands.
 >> > This way Python interpreter will be detected first and based on
 >> > it the Python libraries can be found.
 >> 
 >> > Fixes the following CMake error:
 >> 
 >> > Could NOT find PythonLibs (missing: PYTHON_LIBRARIES PYTHON_INCLUDE_DIRS)
 >> 
 >> We don't seem to have any autobuilder failures in libftdi1 for almost 1
 >> year. Any idea why this doesn't trigger on the autobuilders?

 > In the version 1.4 find_package() calls were made without the REQUIRED
 > keyword. Hence, this error wasn't detected by the autobuilders. This
 > is now fixed upstream and I hope a new release will appear soon. But
 > meanwhile let's apply this fix.

Ahh, so you mean that the python module would just not be built?

That is not what I see here using a defconfig with libftdi1 and python3
on a machine with python 2.7 and python 3.7:

-- Could NOT find Boost
-- Could NOT find Doxygen (missing: DOXYGEN_EXECUTABLE)
-- Not generating API documentation
-- Detected git snapshot version: 2019.11-1302-ge5e84823bb
-- Found SWIG: /home/peko/source/buildroot/output-ftdi/host/bin/swig (found version "4.0.1")
-- Found PythonLibs: /home/peko/source/buildroot/output-ftdi/host/arm-buildroot-linux-uclibcgnueabi/sysroot/usr/lib/libpython3.8.so (found version "3.8.1")
-- Found PythonInterp: /home/peko/source/buildroot/output-ftdi/host/bin/python3.8 (found version "3.8.1")
CMake Deprecation Warning at /usr/share/cmake-3.13/Modules/UseSWIG.cmake:524 (message):
  SWIG_ADD_MODULE is deprecated.  Use SWIG_ADD_LIBRARY instead.
Call Stack (most recent call first):
  python/CMakeLists.txt:28 (swig_add_module)


CMake Warning (dev) at /usr/share/cmake-3.13/Modules/UseSWIG.cmake:564 (message):
  Policy CMP0078 is not set.  Run "cmake --help-policy CMP0078" for policy
  details.  Use the cmake_policy command to set the policy and suppress this
  warning.
Call Stack (most recent call first):
  /usr/share/cmake-3.13/Modules/UseSWIG.cmake:525 (swig_add_library)
  python/CMakeLists.txt:28 (swig_add_module)
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Building python bindings via swig. Will be installed under /usr/lib/python3.8/site-packages


So it correctly finds our host-python3 and does not get confused about
the python 2.7 / 3.7 on my build machine.

In what setup is it failing for you without this patch?

-- 
Bye, Peter Korsgaard

[Buildroot] [PATCH] package/libftdi1: fix python support

[Yegor Yefremov]

On Fri, Feb 7, 2020 at 11:55 AM Peter Korsgaard <peter@korsgaard.com> wrote:
>
> >>>>> "Yegor" == Yegor Yefremov <yegorslists@googlemail.com> writes:
>
>  > Hi Peter,
>  > On Fri, Feb 7, 2020 at 10:13 AM Peter Korsgaard <peter@korsgaard.com> wrote:
>  >>
>  >> >>>>> "yegorslists" == yegorslists  <yegorslists@googlemail.com> writes:
>  >>
>  >> > From: Yegor Yefremov <yegorslists@googlemail.com>
>  >> > Add an upstreamed patch that reorders find_package() commands.
>  >> > This way Python interpreter will be detected first and based on
>  >> > it the Python libraries can be found.
>  >>
>  >> > Fixes the following CMake error:
>  >>
>  >> > Could NOT find PythonLibs (missing: PYTHON_LIBRARIES PYTHON_INCLUDE_DIRS)
>  >>
>  >> We don't seem to have any autobuilder failures in libftdi1 for almost 1
>  >> year. Any idea why this doesn't trigger on the autobuilders?
>
>  > In the version 1.4 find_package() calls were made without the REQUIRED
>  > keyword. Hence, this error wasn't detected by the autobuilders. This
>  > is now fixed upstream and I hope a new release will appear soon. But
>  > meanwhile let's apply this fix.
>
> Ahh, so you mean that the python module would just not be built?
>
> That is not what I see here using a defconfig with libftdi1 and python3
> on a machine with python 2.7 and python 3.7:
>
> -- Could NOT find Boost
> -- Could NOT find Doxygen (missing: DOXYGEN_EXECUTABLE)
> -- Not generating API documentation
> -- Detected git snapshot version: 2019.11-1302-ge5e84823bb
> -- Found SWIG: /home/peko/source/buildroot/output-ftdi/host/bin/swig (found version "4.0.1")
> -- Found PythonLibs: /home/peko/source/buildroot/output-ftdi/host/arm-buildroot-linux-uclibcgnueabi/sysroot/usr/lib/libpython3.8.so (found version "3.8.1")
> -- Found PythonInterp: /home/peko/source/buildroot/output-ftdi/host/bin/python3.8 (found version "3.8.1")
> CMake Deprecation Warning at /usr/share/cmake-3.13/Modules/UseSWIG.cmake:524 (message):
>   SWIG_ADD_MODULE is deprecated.  Use SWIG_ADD_LIBRARY instead.
> Call Stack (most recent call first):
>   python/CMakeLists.txt:28 (swig_add_module)
>
>
> CMake Warning (dev) at /usr/share/cmake-3.13/Modules/UseSWIG.cmake:564 (message):
>   Policy CMP0078 is not set.  Run "cmake --help-policy CMP0078" for policy
>   details.  Use the cmake_policy command to set the policy and suppress this
>   warning.
> Call Stack (most recent call first):
>   /usr/share/cmake-3.13/Modules/UseSWIG.cmake:525 (swig_add_library)
>   python/CMakeLists.txt:28 (swig_add_module)
> This warning is for project developers.  Use -Wno-dev to suppress it.
>
> -- Building python bindings via swig. Will be installed under /usr/lib/python3.8/site-packages
>
>
> So it correctly finds our host-python3 and does not get confused about
> the python 2.7 / 3.7 on my build machine.
>
> In what setup is it failing for you without this patch?

This is what I get when invoking "make libftdi1"

>>> libftdi1 1.4 Configuring
(mkdir -p /home/YegorYefremov/MyProjects/oss/br-test-v2/bsp-uboot/build/libftdi1-1.4/
&& cd /home/YegorYefremov/MyProjects/oss/br-test-v2/bsp-uboot/build/libftdi1-1.4/
&& rm -f CMakeCache.txt &&
PATH="/home/YegorYefremov/MyProjects/oss/br-test-v2/bsp-uboot/host/bin:/home/YegorYefremov/MyProjects/oss/br-test-v2/bsp-uboot/host/sbin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games"
 /usr/bin/cmake
/home/YegorYefremov/MyProjects/oss/br-test-v2/bsp-uboot/build/libftdi1-1.4/
-DCMAKE_TOOLCHAIN_FILE="/home/YegorYefremov/MyProjects/oss/br-test-v2/bsp-uboot/host/share/buildroot/toolchainfile.cmake"
-DCMAKE_INSTALL_PREFIX="/usr" -DCMAKE_COLOR_MAKEFILE=OFF
-DBUILD_DOC=OFF -DBUILD_DOCS=OFF -DBUILD_EXAMPLE=OFF
-DBUILD_EXAMPLES=OFF -DBUILD_TEST=OFF -DBUILD_TESTS=OFF
-DBUILD_TESTING=OFF -DBUILD_SHARED_LIBS=ON  -DDOCUMENTATION=OFF
-DEXAMPLES=OFF -DFTDIPP=OFF -DPYTHON_BINDINGS=ON -DFTDI_EEPROM=OFF )
-- The C compiler identification is GNU 9.2.1
-- Check for working C compiler:
/home/user/br-test-v2/bsp-uboot/host/bin/arm-none-linux-gnueabihf-gcc
-- Check for working C compiler:
/home/user/br-test-v2/bsp-uboot/host/bin/arm-none-linux-gnueabihf-gcc
-- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Build type: Release
-- Found PkgConfig:
/home/user/br-test-v2/bsp-uboot/host/bin/pkg-config (found version
"1.6.1")
-- Checking for module 'libusb-1.0'
--   Found libusb-1.0, version 1.0.23
-- Found LIBUSB:
/home/user/br-test-v2/bsp-uboot/host/arm-buildroot-linux-gnueabihf/sysroot/usr/lib/libusb-1.0.so
-- Could NOT find Boost
-- Found Doxygen: /usr/bin/doxygen (found version "1.8.13") found
components:  doxygen dot
-- Not generating API documentation
-- Detected git snapshot version: 2019.11-1302-ge5e84823bb
-- Found SWIG: /usr/bin/swig3.0 (found version "3.0.12")
-- Could NOT find PythonLibs (missing: PYTHON_LIBRARIES PYTHON_INCLUDE_DIRS)
-- Found PythonInterp: /home/user/br-test-v2/bsp-uboot/host/bin/python
(found version "3.8.1")
-- Not building python bindings
-- ftdi_eeprom build is disabled
-- libConfuse not found, won't build ftdi_eeprom
-- Not building example programs.
-- NOT building unit test
-- Configuring done
-- Generating done

I was using this defconfig [1]. To speed up the build I've changed the
ABI to ARMHF and used the default external toolchain. And python3,
libftdi1 together with python bindings were selected.

My host has:

Python 3.6.7
Python 2.7.15rc1

[1] https://github.com/visionsystemsgmbh/onrisc_br_bsp/blob/master/configs/baltos_defconfig

P.S. another issue is that as you can see cmake has found the host's
own SWIG version and not BR's version which is 4.0.x.

Yegor

[Buildroot] [PATCH] package/libftdi1: fix python support

[Peter Korsgaard]

>>>>> "Yegor" == Yegor Yefremov <yegorslists@googlemail.com> writes:

Hi,

 > This is what I get when invoking "make libftdi1"

Ok. It is not really clear to me why it fails for you but not for me,
but ok - Committed, thanks.

 > P.S. another issue is that as you can see cmake has found the host's
 > own SWIG version and not BR's version which is 4.0.x.

Hmm, indeed. We do add our host-swig to the path, so I guess something
is hardcoded to look in /usr/bin first?

Care to take a look?

-- 
Bye, Peter Korsgaard

[Buildroot] [PATCH] package/libftdi1: fix python support

[Yegor Yefremov]

On Fri, Feb 7, 2020 at 3:53 PM Peter Korsgaard <peter@korsgaard.com> wrote:
>
> >>>>> "Yegor" == Yegor Yefremov <yegorslists@googlemail.com> writes:
>
> Hi,
>
>  > This is what I get when invoking "make libftdi1"
>
> Ok. It is not really clear to me why it fails for you but not for me,
> but ok - Committed, thanks.
>
>  > P.S. another issue is that as you can see cmake has found the host's
>  > own SWIG version and not BR's version which is 4.0.x.
>
> Hmm, indeed. We do add our host-swig to the path, so I guess something
> is hardcoded to look in /usr/bin first?
>
> Care to take a look?

Will do.

Yegor

[Buildroot] [PATCH] package/libftdi1: fix python support

[Yegor Yefremov]

Hi Peter,

On Fri, Feb 7, 2020 at 3:55 PM Yegor Yefremov
<yegorslists@googlemail.com> wrote:
>
> On Fri, Feb 7, 2020 at 3:53 PM Peter Korsgaard <peter@korsgaard.com> wrote:
> >
> > >>>>> "Yegor" == Yegor Yefremov <yegorslists@googlemail.com> writes:
> >
> > Hi,
> >
> >  > This is what I get when invoking "make libftdi1"
> >
> > Ok. It is not really clear to me why it fails for you but not for me,
> > but ok - Committed, thanks.
> >
> >  > P.S. another issue is that as you can see cmake has found the host's
> >  > own SWIG version and not BR's version which is 4.0.x.
> >
> > Hmm, indeed. We do add our host-swig to the path, so I guess something
> > is hardcoded to look in /usr/bin first?
> >
> > Care to take a look?
>
> Will do.

I have looked at the problem. CMake 3.10.2 doesn't know anything about
SWIG4.0 so it looks for the SWIG binaries in this order (see
Modules/FindSWIG.cmake):

find_program(SWIG_EXECUTABLE NAMES swig3.0 swig2.0 swig)

This way swig3.0 on our host will be detected first. I've added
swig4.0 to my "/usr/share/cmake-3.10/Modules/FindSWIG.cmake" and it
now detects the SWIG built with BR. So for now, the only solution is
either to install an up-to-date CMake via "pip install cmake" or to
bump our CMake requirement to a newer version.

This is the reference of the commit adding SWIG4.0 support: Refs:
v3.14.3-1-g65c5c1b567 [1].

[1] https://github.com/Kitware/CMake/commit/65c5c1b567bdeb4c33ac2f6ad92786c159edf82a

Yegor

[Buildroot] [PATCH] package/libftdi1: fix python support

[Peter Korsgaard]

>>>>> "Yegor" == Yegor Yefremov <yegorslists@googlemail.com> writes:

Hi,

 >> >  > P.S. another issue is that as you can see cmake has found the host's
 >> >  > own SWIG version and not BR's version which is 4.0.x.
 >> >
 >> > Hmm, indeed. We do add our host-swig to the path, so I guess something
 >> > is hardcoded to look in /usr/bin first?
 >> >
 >> > Care to take a look?
 >> 
 >> Will do.

 > I have looked at the problem. CMake 3.10.2 doesn't know anything about
 > SWIG4.0 so it looks for the SWIG binaries in this order (see
 > Modules/FindSWIG.cmake):

 > find_program(SWIG_EXECUTABLE NAMES swig3.0 swig2.0 swig)

 > This way swig3.0 on our host will be detected first. I've added
 > swig4.0 to my "/usr/share/cmake-3.10/Modules/FindSWIG.cmake" and it
 > now detects the SWIG built with BR. So for now, the only solution is
 > either to install an up-to-date CMake via "pip install cmake" or to
 > bump our CMake requirement to a newer version.

Or if swig is backwards compatible (I guess it is?), we could just add a
swig3.0 symlink as well?

-- 
Bye, Peter Korsgaard

[Buildroot] [PATCH] package/libftdi1: fix python support

[Yegor Yefremov]

On Sat, Feb 15, 2020 at 2:02 PM Peter Korsgaard <peter@korsgaard.com> wrote:
>
> >>>>> "Yegor" == Yegor Yefremov <yegorslists@googlemail.com> writes:
>
> Hi,
>
>  >> >  > P.S. another issue is that as you can see cmake has found the host's
>  >> >  > own SWIG version and not BR's version which is 4.0.x.
>  >> >
>  >> > Hmm, indeed. We do add our host-swig to the path, so I guess something
>  >> > is hardcoded to look in /usr/bin first?
>  >> >
>  >> > Care to take a look?
>  >>
>  >> Will do.
>
>  > I have looked at the problem. CMake 3.10.2 doesn't know anything about
>  > SWIG4.0 so it looks for the SWIG binaries in this order (see
>  > Modules/FindSWIG.cmake):
>
>  > find_program(SWIG_EXECUTABLE NAMES swig3.0 swig2.0 swig)
>
>  > This way swig3.0 on our host will be detected first. I've added
>  > swig4.0 to my "/usr/share/cmake-3.10/Modules/FindSWIG.cmake" and it
>  > now detects the SWIG built with BR. So for now, the only solution is
>  > either to install an up-to-date CMake via "pip install cmake" or to
>  > bump our CMake requirement to a newer version.
>
> Or if swig is backwards compatible (I guess it is?), we could just add a
> swig3.0 symlink as well?

Sure. Will take care of this.

Yegor