Re: [PATCH V2] distutils/setuptools-native, distutils3/setuptools3-native: do not try to fetch code during do_compile

[Martin Jansa <martin.jansa@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 7068 bytes --]

Thanks a lot.

On Fri, Jul 27, 2018 at 2:43 PM Derek Straka <derek@asterius.io> wrote:

> python-daemon - https://patchwork.openembedded.org/patch/153034/
>
> python-stevedore - https://patchwork.openembedded.org/patch/153032/
>
> I think that's everything though my world python build is still going.
> Thanks for pointing the failures out and let me know if there are others.
> I'm guessing this will ripple through the python heavy layers; I found a
> couple yesterday in meta-iot-cloud.
>
> On Fri, Jul 27, 2018 at 7:41 AM, Derek Straka <derek@asterius.io> wrote:
>
>> Yes.  I was just waiting for my master build to wrap up.
>>
>> On Fri, Jul 27, 2018, 7:39 AM Martin Jansa <martin.jansa@gmail.com>
>> wrote:
>>
>>> I like this change, some recipes in meta-oe seems to be broken now, is
>>> someone already working on a fix?
>>>
>>> I'm seeing following 5 failures in my world builds:
>>>
>>> python-stevedore-1.28.0-r0 do_compile and python3-stevedore-1.28.0-r0
>>> do_compile
>>> pkg_resources.DistributionNotFound: The 'pbr>=2.0.0' distribution was
>>> not found and is required by the application
>>>
>>> python-dateutil-2.7.3-r0 do_compile and python3-dateutil-2.7.3-r0
>>> do_compile
>>> pkg_resources.DistributionNotFound: The 'setuptools_scm' distribution
>>> was not found and is required by the application
>>>
>>> python-daemon-2.1.2-r0 do_compile
>>> pkg_resources.DistributionNotFound: The 'docutils' distribution was not
>>> found and is required by the application
>>>
>>> I've found a fix for python-dateutils from Derek:
>>> https://patchwork.openembedded.org/patch/153002/
>>>
>>> On Tue, Jul 17, 2018 at 3:27 PM Hongxu Jia <hongxu.jia@windriver.com>
>>> wrote:
>>>
>>>> If a python recipe is using setuptools and the setup_requires argument,
>>>> where
>>>> setuptools will use easy_install to fetch the module if it isn't
>>>> present.
>>>>
>>>> The build failed on a machine where a proxy was required, but succeeded
>>>> on a
>>>> machine which had direct access to the internet
>>>>
>>>> Add var-NO_FETCH_BUILD, and set it in distutils_do_compile which does
>>>> not
>>>> allow to fetch code from internet during do_compile.
>>>>
>>>> Example result:
>>>> ...
>>>> ERROR: Do not try to fetch `pytest-runner1' for building. Please add
>>>> its native recipe to DEPENDS.
>>>> Traceback (most recent call last):
>>>>   File "setup.py", line 56, in <module>
>>>> ...
>>>>
>>>> The improvement is flexible for test_requirements argument (used at
>>>> `setup.py test'), where use easy_install also.
>>>>
>>>> [YOCTO #12084]
>>>> Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
>>>> ---
>>>>  meta/classes/distutils.bbclass                     |  1 +
>>>>  meta/classes/distutils3.bbclass                    |  1 +
>>>>  ...ionally-do-not-fetch-code-by-easy_install.patch | 34
>>>> ++++++++++++++++++++++
>>>>  meta/recipes-devtools/python/python-setuptools.inc |  2 ++
>>>>  4 files changed, 38 insertions(+)
>>>>  create mode 100644
>>>> meta/recipes-devtools/python/files/0001-conditionally-do-not-fetch-code-by-easy_install.patch
>>>>
>>>> diff --git a/meta/classes/distutils.bbclass
>>>> b/meta/classes/distutils.bbclass
>>>> index 1930c35..c2f7403 100644
>>>> --- a/meta/classes/distutils.bbclass
>>>> +++ b/meta/classes/distutils.bbclass
>>>> @@ -8,6 +8,7 @@ DISTUTILS_INSTALL_ARGS ?= "--prefix=${D}/${prefix} \
>>>>      --install-data=${D}/${datadir}"
>>>>
>>>>  distutils_do_compile() {
>>>> +         NO_FETCH_BUILD=1 \
>>>>           STAGING_INCDIR=${STAGING_INCDIR} \
>>>>           STAGING_LIBDIR=${STAGING_LIBDIR} \
>>>>           ${STAGING_BINDIR_NATIVE}/${PYTHON_PN}-native/${PYTHON_PN}
>>>> setup.py build ${DISTUTILS_BUILD_ARGS} || \
>>>> diff --git a/meta/classes/distutils3.bbclass
>>>> b/meta/classes/distutils3.bbclass
>>>> index d4b92a3..f8aca5e 100644
>>>> --- a/meta/classes/distutils3.bbclass
>>>> +++ b/meta/classes/distutils3.bbclass
>>>> @@ -15,6 +15,7 @@ distutils3_do_configure() {
>>>>  }
>>>>
>>>>  distutils3_do_compile() {
>>>> +        NO_FETCH_BUILD=1 \
>>>>          STAGING_INCDIR=${STAGING_INCDIR} \
>>>>          STAGING_LIBDIR=${STAGING_LIBDIR} \
>>>>          ${STAGING_BINDIR_NATIVE}/${PYTHON_PN}-native/${PYTHON_PN}
>>>> setup.py \
>>>> diff --git
>>>> a/meta/recipes-devtools/python/files/0001-conditionally-do-not-fetch-code-by-easy_install.patch
>>>> b/meta/recipes-devtools/python/files/0001-conditionally-do-not-fetch-code-by-easy_install.patch
>>>> new file mode 100644
>>>> index 0000000..4b17921
>>>> --- /dev/null
>>>> +++
>>>> b/meta/recipes-devtools/python/files/0001-conditionally-do-not-fetch-code-by-easy_install.patch
>>>> @@ -0,0 +1,34 @@
>>>> +From 019422d19c6401fa8f45b1311ecd31bb087c809d Mon Sep 17 00:00:00 2001
>>>> +From: Hongxu Jia <hongxu.jia@windriver.com>
>>>> +Date: Tue, 17 Jul 2018 10:13:38 +0800
>>>> +Subject: [PATCH] conditionally do not fetch code by easy_install
>>>> +
>>>> +If var-NO_FETCH_BUILD is set, do not allow to fetch code from
>>>> +internet by easy_install.
>>>> +
>>>> +Upstream-Status: Inappropriate [oe specific]
>>>> +
>>>> +Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
>>>> +---
>>>> + setuptools/command/easy_install.py | 5 +++++
>>>> + 1 file changed, 5 insertions(+)
>>>> +
>>>> +diff --git a/setuptools/command/easy_install.py
>>>> b/setuptools/command/easy_install.py
>>>> +index 85ee40f..7eb9d2a 100755
>>>> +--- a/setuptools/command/easy_install.py
>>>> ++++ b/setuptools/command/easy_install.py
>>>> +@@ -637,6 +637,11 @@ class easy_install(Command):
>>>> +             os.path.exists(tmpdir) and rmtree(rmtree_safe(tmpdir))
>>>> +
>>>> +     def easy_install(self, spec, deps=False):
>>>> ++        if os.environ.get('NO_FETCH_BUILD', None):
>>>> ++            log.error("ERROR: Do not try to fetch `%s' for building. "
>>>> ++                      "Please add its native recipe to DEPENDS." %
>>>> spec)
>>>> ++            return None
>>>> ++
>>>> +         if not self.editable:
>>>> +             self.install_site_py()
>>>> +
>>>> +--
>>>> +2.7.4
>>>> +
>>>> diff --git a/meta/recipes-devtools/python/python-setuptools.inc
>>>> b/meta/recipes-devtools/python/python-setuptools.inc
>>>> index 25be446..24b3ac4 100644
>>>> --- a/meta/recipes-devtools/python/python-setuptools.inc
>>>> +++ b/meta/recipes-devtools/python/python-setuptools.inc
>>>> @@ -9,6 +9,8 @@ PYPI_PACKAGE_EXT = "zip"
>>>>
>>>>  inherit pypi
>>>>
>>>> +SRC_URI_append_class-native = "
>>>> file://0001-conditionally-do-not-fetch-code-by-easy_install.patch"
>>>> +
>>>>  SRC_URI[md5sum] = "260630ae1a64bafa39dcc53404d63829"
>>>>  SRC_URI[sha256sum] =
>>>> "012adb8e25fbfd64c652e99e7bab58799a3aaf05d39ab38561f69190a909015f"
>>>>
>>>> --
>>>> 2.7.4
>>>>
>>>> --
>>>> _______________________________________________
>>>> Openembedded-core mailing list
>>>> Openembedded-core@lists.openembedded.org
>>>> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>>>>
>>>
>

[-- Attachment #2: Type: text/html, Size: 9107 bytes --]