* Python module and build_ext args
@ 2017-08-25 1:24 Craig McQueen
2017-08-25 1:47 ` Craig McQueen
0 siblings, 1 reply; 7+ messages in thread
From: Craig McQueen @ 2017-08-25 1:24 UTC (permalink / raw)
To: yocto
I'm trying to make a recipe for python3-uvloop, using setuptools3.
The Python 3 uvloop module depends on libuv. It bundles a version of libuv, and setup.py tries to build it, but it doesn't work well for cross-compilation. However, it also provides a build_ext parameter "--use-system-libuv", which seems to work when I try running it manually in devshell. (I have made a suitable recipe for libuv and added libuv to DEPENDS.)
How can I specify the "--use-system-libuv" parameter for build_ext in the python3-uvloop recipe? I see a reference to DISTUTILS_BUILD_EXT_ARGS, but it doesn't seem to be functional.
--
Craig McQueen
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Python module and build_ext args
2017-08-25 1:24 Python module and build_ext args Craig McQueen
@ 2017-08-25 1:47 ` Craig McQueen
2017-08-25 2:41 ` Khem Raj
0 siblings, 1 reply; 7+ messages in thread
From: Craig McQueen @ 2017-08-25 1:47 UTC (permalink / raw)
To: Craig McQueen, yocto
I wrote:
> I'm trying to make a recipe for python3-uvloop, using setuptools3.
>
> The Python 3 uvloop module depends on libuv. It bundles a version of libuv,
> and setup.py tries to build it, but it doesn't work well for cross-compilation.
> However, it also provides a build_ext parameter "--use-system-libuv", which
> seems to work when I try running it manually in devshell. (I have made a
> suitable recipe for libuv and added libuv to DEPENDS.)
>
> How can I specify the "--use-system-libuv" parameter for build_ext in the
> python3-uvloop recipe? I see a reference to DISTUTILS_BUILD_EXT_ARGS,
> but it doesn't seem to be functional.
I see DISTUTILS_BUILD_EXT_ARGS was submitted in this patch:
https://patchwork.openembedded.org/patch/66071/
However, in the Yocto poky repository, I see commit 0221af0f4ee9e8bfb8796841bdf806e38bc600c6 which appears to be a broken version of the above patch with the separate build_ext step not actually executed with the DISTUTILS_BUILD_EXT_ARGS parameters.
--
Craig McQueen
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Python module and build_ext args
2017-08-25 1:47 ` Craig McQueen
@ 2017-08-25 2:41 ` Khem Raj
2017-08-25 4:42 ` Craig McQueen
0 siblings, 1 reply; 7+ messages in thread
From: Khem Raj @ 2017-08-25 2:41 UTC (permalink / raw)
To: Craig McQueen, yocto
[-- Attachment #1: Type: text/plain, Size: 1398 bytes --]
On Thu, Aug 24, 2017 at 6:48 PM Craig McQueen <craig.mcqueen@innerrange.com>
wrote:
> I wrote:
> > I'm trying to make a recipe for python3-uvloop, using setuptools3.
> >
> > The Python 3 uvloop module depends on libuv. It bundles a version of
> libuv,
> > and setup.py tries to build it, but it doesn't work well for
> cross-compilation.
> > However, it also provides a build_ext parameter "--use-system-libuv",
> which
> > seems to work when I try running it manually in devshell. (I have made a
> > suitable recipe for libuv and added libuv to DEPENDS.)
> >
> > How can I specify the "--use-system-libuv" parameter for build_ext in the
> > python3-uvloop recipe? I see a reference to DISTUTILS_BUILD_EXT_ARGS,
> > but it doesn't seem to be functional.
>
>
> I see DISTUTILS_BUILD_EXT_ARGS was submitted in this patch:
> https://patchwork.openembedded.org/patch/66071/
>
> However, in the Yocto poky repository, I see commit
> 0221af0f4ee9e8bfb8796841bdf806e38bc600c6 which appears to be a broken
> version of the above patch with the separate build_ext step not actually
> executed with the DISTUTILS_BUILD_EXT_ARGS parameters.
You did not explain broken in which sense ?
>
>
> --
> Craig McQueen
>
> --
> _______________________________________________
> yocto mailing list
> yocto@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/yocto
>
[-- Attachment #2: Type: text/html, Size: 2141 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Python module and build_ext args
2017-08-25 2:41 ` Khem Raj
@ 2017-08-25 4:42 ` Craig McQueen
2017-08-25 4:45 ` Khem Raj
0 siblings, 1 reply; 7+ messages in thread
From: Craig McQueen @ 2017-08-25 4:42 UTC (permalink / raw)
To: Khem Raj, yocto
[-- Attachment #1: Type: text/plain, Size: 2272 bytes --]
Khem Raj wrote:
On Thu, Aug 24, 2017 at 6:48 PM Craig McQueen <craig.mcqueen@innerrange.com<mailto:craig.mcqueen@innerrange.com>> wrote:
I wrote:
> I'm trying to make a recipe for python3-uvloop, using setuptools3.
>
> The Python 3 uvloop module depends on libuv. It bundles a version of libuv,
> and setup.py tries to build it, but it doesn't work well for cross-compilation.
> However, it also provides a build_ext parameter "--use-system-libuv", which
> seems to work when I try running it manually in devshell. (I have made a
> suitable recipe for libuv and added libuv to DEPENDS.)
>
> How can I specify the "--use-system-libuv" parameter for build_ext in the
> python3-uvloop recipe? I see a reference to DISTUTILS_BUILD_EXT_ARGS,
> but it doesn't seem to be functional.
I see DISTUTILS_BUILD_EXT_ARGS was submitted in this patch:
https://patchwork.openembedded.org/patch/66071/
However, in the Yocto poky repository, I see commit 0221af0f4ee9e8bfb8796841bdf806e38bc600c6 which appears to be a broken version of the above patch with the separate build_ext step not actually executed with the DISTUTILS_BUILD_EXT_ARGS parameters.
You did not explain broken in which sense ?
It is broken in the sense that: The separate build_ext step is not actually executed with the DISTUTILS_BUILD_EXT_ARGS parameters. The original patch submission contained:
STAGING_INCDIR=${STAGING_INCDIR} \
STAGING_LIBDIR=${STAGING_LIBDIR} \
BUILD_SYS=${BUILD_SYS} HOST_SYS=${HOST_SYS} \
- ${STAGING_BINDIR_NATIVE}/${PYTHON_PN}-native/${PYTHON_PN} setup.py build ${DISTUTILS_BUILD_ARGS} || \
+ ${STAGING_BINDIR_NATIVE}/${PYTHON_PN}-native/${PYTHON_PN} setup.py \
+ build_ext --include-dirs ${STAGING_INCDIR}/${PYTHON_DIR}${PYTHON_ABI} \
+ --library-dirs ${STAGING_LIBCDIR}/${PYTHON_DIR} \
+ ${DISTUTILS_BUILD_EXT_ARGS} \
+ build ${DISTUTILS_BUILD_ARGS} || \
+ ${STAGING_BINDIR_NATIVE}/${PYTHON_PN}-native/${PYTHON_PN} setup.py build_ext ${DISTUTILS_BUILD_ARGS} || \
bbfatal "${PYTHON_PN} setup.py build_ext execution failed."
}
However that part of the patch is not present in commit 0221af0f4ee9e8bfb8796841bdf806e38bc600c6.
--
Craig McQueen
[-- Attachment #2: Type: text/html, Size: 7898 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Python module and build_ext args
2017-08-25 4:42 ` Craig McQueen
@ 2017-08-25 4:45 ` Khem Raj
2017-08-29 2:38 ` Craig McQueen
0 siblings, 1 reply; 7+ messages in thread
From: Khem Raj @ 2017-08-25 4:45 UTC (permalink / raw)
To: Craig McQueen; +Cc: yocto
On Thu, Aug 24, 2017 at 9:42 PM, Craig McQueen
<craig.mcqueen@innerrange.com> wrote:
> Khem Raj wrote:
>
>
>
> On Thu, Aug 24, 2017 at 6:48 PM Craig McQueen <craig.mcqueen@innerrange.com>
> wrote:
>
> I wrote:
>> I'm trying to make a recipe for python3-uvloop, using setuptools3.
>>
>> The Python 3 uvloop module depends on libuv. It bundles a version of
>> libuv,
>> and setup.py tries to build it, but it doesn't work well for
>> cross-compilation.
>> However, it also provides a build_ext parameter "--use-system-libuv",
>> which
>> seems to work when I try running it manually in devshell. (I have made a
>> suitable recipe for libuv and added libuv to DEPENDS.)
>>
>> How can I specify the "--use-system-libuv" parameter for build_ext in the
>> python3-uvloop recipe? I see a reference to DISTUTILS_BUILD_EXT_ARGS,
>> but it doesn't seem to be functional.
>
>
> I see DISTUTILS_BUILD_EXT_ARGS was submitted in this patch:
> https://patchwork.openembedded.org/patch/66071/
>
> However, in the Yocto poky repository, I see commit
> 0221af0f4ee9e8bfb8796841bdf806e38bc600c6 which appears to be a broken
> version of the above patch with the separate build_ext step not actually
> executed with the DISTUTILS_BUILD_EXT_ARGS parameters.
>
>
>
> You did not explain broken in which sense ?
>
>
>
> It is broken in the sense that: The separate build_ext step is not actually
> executed with the DISTUTILS_BUILD_EXT_ARGS parameters. The original patch
> submission contained:
>
>
>
> STAGING_INCDIR=${STAGING_INCDIR} \
>
> STAGING_LIBDIR=${STAGING_LIBDIR} \
>
> BUILD_SYS=${BUILD_SYS} HOST_SYS=${HOST_SYS} \
>
> - ${STAGING_BINDIR_NATIVE}/${PYTHON_PN}-native/${PYTHON_PN} setup.py
> build ${DISTUTILS_BUILD_ARGS} || \
>
> + ${STAGING_BINDIR_NATIVE}/${PYTHON_PN}-native/${PYTHON_PN} setup.py
> \
>
> + build_ext --include-dirs
> ${STAGING_INCDIR}/${PYTHON_DIR}${PYTHON_ABI} \
>
> + --library-dirs ${STAGING_LIBCDIR}/${PYTHON_DIR} \
>
> + ${DISTUTILS_BUILD_EXT_ARGS} \
>
> + build ${DISTUTILS_BUILD_ARGS} || \
>
> + ${STAGING_BINDIR_NATIVE}/${PYTHON_PN}-native/${PYTHON_PN} setup.py
> build_ext ${DISTUTILS_BUILD_ARGS} || \
>
> bbfatal "${PYTHON_PN} setup.py build_ext execution failed."
>
> }
>
>
>
> However that part of the patch is not present in commit
> 0221af0f4ee9e8bfb8796841bdf806e38bc600c6.
>
IIRC there were some breakages with this but it was long time ago. Can
you appy this change and test it out ?
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Python module and build_ext args
2017-08-25 4:45 ` Khem Raj
@ 2017-08-29 2:38 ` Craig McQueen
2017-09-01 1:41 ` Craig McQueen
0 siblings, 1 reply; 7+ messages in thread
From: Craig McQueen @ 2017-08-29 2:38 UTC (permalink / raw)
To: Khem Raj; +Cc: yocto
Khem Raj wrote:
> On Thu, Aug 24, 2017 at 9:42 PM, Craig McQueen
> <craig.mcqueen@innerrange.com> wrote:
> > Khem Raj wrote:
> > On Thu, Aug 24, 2017 at 6:48 PM Craig McQueen
> > <craig.mcqueen@innerrange.com>
> > wrote:
> >
> > I wrote:
> >> I'm trying to make a recipe for python3-uvloop, using setuptools3.
> >>
> >> The Python 3 uvloop module depends on libuv. It bundles a version of
> >> libuv, and setup.py tries to build it, but it doesn't work well for
> >> cross-compilation.
> >> However, it also provides a build_ext parameter "--use-system-libuv",
> >> which seems to work when I try running it manually in devshell. (I
> >> have made a suitable recipe for libuv and added libuv to DEPENDS.)
> >>
> >> How can I specify the "--use-system-libuv" parameter for build_ext in
> >> the python3-uvloop recipe? I see a reference to
> >> DISTUTILS_BUILD_EXT_ARGS, but it doesn't seem to be functional.
> >
> >
> > I see DISTUTILS_BUILD_EXT_ARGS was submitted in this patch:
> > https://patchwork.openembedded.org/patch/66071/
> >
> > However, in the Yocto poky repository, I see commit
> > 0221af0f4ee9e8bfb8796841bdf806e38bc600c6 which appears to be a
> broken
> > version of the above patch with the separate build_ext step not
> > actually executed with the DISTUTILS_BUILD_EXT_ARGS parameters.
> >
> >
> >
> > You did not explain broken in which sense ?
> >
> >
> >
> > It is broken in the sense that: The separate build_ext step is not
> > actually executed with the DISTUTILS_BUILD_EXT_ARGS parameters. The
> > original patch submission contained:
> >
> >
> >
> > STAGING_INCDIR=${STAGING_INCDIR} \
> >
> > STAGING_LIBDIR=${STAGING_LIBDIR} \
> >
> > BUILD_SYS=${BUILD_SYS} HOST_SYS=${HOST_SYS} \
> >
> > - ${STAGING_BINDIR_NATIVE}/${PYTHON_PN}-native/${PYTHON_PN}
> setup.py
> > build ${DISTUTILS_BUILD_ARGS} || \
> >
> > + ${STAGING_BINDIR_NATIVE}/${PYTHON_PN}-native/${PYTHON_PN}
> > + setup.py
> > \
> >
> > + build_ext --include-dirs
> > ${STAGING_INCDIR}/${PYTHON_DIR}${PYTHON_ABI} \
> >
> > + --library-dirs ${STAGING_LIBCDIR}/${PYTHON_DIR} \
> >
> > + ${DISTUTILS_BUILD_EXT_ARGS} \
> >
> > + build ${DISTUTILS_BUILD_ARGS} || \
> >
> > + ${STAGING_BINDIR_NATIVE}/${PYTHON_PN}-native/${PYTHON_PN}
> > + setup.py
> > build_ext ${DISTUTILS_BUILD_ARGS} || \
> >
> > bbfatal "${PYTHON_PN} setup.py build_ext execution failed."
> >
> > }
> >
> >
> >
> > However that part of the patch is not present in commit
> > 0221af0f4ee9e8bfb8796841bdf806e38bc600c6.
> >
>
> IIRC there were some breakages with this but it was long time ago. Can you
> appy this change and test it out ?
I tried applying the change manually. The Yocto do_compile step succeeded. But then the do_install step still failed. It seems that the do_install step (which runs setup.py install) still runs the setup.py build_ext step a second time, without the extra option I was trying to use.
I'm not familiar enough with the working of python3 setup.py to know why the install step re-runs the build_ext step a second time.
--
Craig McQueen
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Python module and build_ext args
2017-08-29 2:38 ` Craig McQueen
@ 2017-09-01 1:41 ` Craig McQueen
0 siblings, 0 replies; 7+ messages in thread
From: Craig McQueen @ 2017-09-01 1:41 UTC (permalink / raw)
To: Craig McQueen, Khem Raj; +Cc: yocto
I wrote:
> Khem Raj wrote:
> > On Thu, Aug 24, 2017 at 9:42 PM, Craig McQueen
> > <craig.mcqueen@innerrange.com> wrote:
> > > Khem Raj wrote:
> > > On Thu, Aug 24, 2017 at 6:48 PM Craig McQueen
> > > <craig.mcqueen@innerrange.com>
> > > wrote:
> > >
> > > I wrote:
> > >> I'm trying to make a recipe for python3-uvloop, using setuptools3.
> > >>
> > >> The Python 3 uvloop module depends on libuv. It bundles a version
> > >> of libuv, and setup.py tries to build it, but it doesn't work well
> > >> for cross-compilation.
> > >> However, it also provides a build_ext parameter
> > >> "--use-system-libuv", which seems to work when I try running it
> > >> manually in devshell. (I have made a suitable recipe for libuv and
> > >> added libuv to DEPENDS.)
> > >>
> > >> How can I specify the "--use-system-libuv" parameter for build_ext
> > >> in the python3-uvloop recipe? I see a reference to
> > >> DISTUTILS_BUILD_EXT_ARGS, but it doesn't seem to be functional.
> > >
> > >
> > > I see DISTUTILS_BUILD_EXT_ARGS was submitted in this patch:
> > > https://patchwork.openembedded.org/patch/66071/
> > >
> > > However, in the Yocto poky repository, I see commit
> > > 0221af0f4ee9e8bfb8796841bdf806e38bc600c6 which appears to be a
> > broken
> > > version of the above patch with the separate build_ext step not
> > > actually executed with the DISTUTILS_BUILD_EXT_ARGS parameters.
> > >
> > >
> > >
> > > You did not explain broken in which sense ?
> > >
> > >
> > >
> > > It is broken in the sense that: The separate build_ext step is not
> > > actually executed with the DISTUTILS_BUILD_EXT_ARGS parameters. The
> > > original patch submission contained:
> > >
> > >
> > >
> > > STAGING_INCDIR=${STAGING_INCDIR} \
> > >
> > > STAGING_LIBDIR=${STAGING_LIBDIR} \
> > >
> > > BUILD_SYS=${BUILD_SYS} HOST_SYS=${HOST_SYS} \
> > >
> > > - ${STAGING_BINDIR_NATIVE}/${PYTHON_PN}-
> native/${PYTHON_PN}
> > setup.py
> > > build ${DISTUTILS_BUILD_ARGS} || \
> > >
> > > + ${STAGING_BINDIR_NATIVE}/${PYTHON_PN}-
> native/${PYTHON_PN}
> > > + setup.py
> > > \
> > >
> > > + build_ext --include-dirs
> > > ${STAGING_INCDIR}/${PYTHON_DIR}${PYTHON_ABI} \
> > >
> > > + --library-dirs ${STAGING_LIBCDIR}/${PYTHON_DIR} \
> > >
> > > + ${DISTUTILS_BUILD_EXT_ARGS} \
> > >
> > > + build ${DISTUTILS_BUILD_ARGS} || \
> > >
> > > + ${STAGING_BINDIR_NATIVE}/${PYTHON_PN}-
> native/${PYTHON_PN}
> > > + setup.py
> > > build_ext ${DISTUTILS_BUILD_ARGS} || \
> > >
> > > bbfatal "${PYTHON_PN} setup.py build_ext execution failed."
> > >
> > > }
> > >
> > >
> > >
> > > However that part of the patch is not present in commit
> > > 0221af0f4ee9e8bfb8796841bdf806e38bc600c6.
> > >
> >
> > IIRC there were some breakages with this but it was long time ago. Can
> > you appy this change and test it out ?
>
> I tried applying the change manually. The Yocto do_compile step succeeded.
> But then the do_install step still failed. It seems that the do_install step
> (which runs setup.py install) still runs the setup.py build_ext step a second
> time, without the extra option I was trying to use.
>
> I'm not familiar enough with the working of python3 setup.py to know why
> the install step re-runs the build_ext step a second time.
I wasn't able to find an elegant solution for DISTUTILS_BUILD_EXT_ARGS. I ended up getting a working python3-uvloop recipe a different way, using two patches to the setup.py.
I have posted to the uvloop project regarding this:
https://github.com/MagicStack/uvloop/issues/104
--
Craig McQueen
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2017-09-01 1:41 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-08-25 1:24 Python module and build_ext args Craig McQueen
2017-08-25 1:47 ` Craig McQueen
2017-08-25 2:41 ` Khem Raj
2017-08-25 4:42 ` Craig McQueen
2017-08-25 4:45 ` Khem Raj
2017-08-29 2:38 ` Craig McQueen
2017-09-01 1:41 ` Craig McQueen
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.