[Buildroot] [Bug 13321] New: utils/scanpypi: fails with packages that define a package’s metadata in setup.cfg

[Buildroot] [Bug 13321] New: utils/scanpypi: fails with packages that define a package’s metadata in setup.cfg

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=13321

            Bug ID: 13321
           Summary: utils/scanpypi: fails with packages that define a
                    package?s metadata in setup.cfg
           Product: buildroot
           Version: 2020.08.2
          Hardware: All
                OS: Linux
            Status: NEW
          Severity: normal
          Priority: P5
         Component: Other
          Assignee: unassigned at buildroot.uclibc.org
          Reporter: bartosz.bilas at hotmail.com
                CC: buildroot at uclibc.org
  Target Milestone: ---

There is a couple(or more than couple) of packages on PyPi that doesn't work
with the scanpypi script (for example wheel/importlib_resources). The reason of
that is that it uses setup.cfg in conjunction with setup.py to package metadata
configuration. Use of scanpypi on these packages fails with the following
errors:

utils/scanpypi:23: DeprecationWarning: the imp module is deprecated in favour
of importlib; see the module's documentation for alternative uses
  import imp
spdx_lookup module is not installed. This can lead to an inaccurate licence
detection. Please install it via
pip install spdx_lookup
buildroot package name for wheel: python-wheel
Package: python-wheel
Fetching package wheel
Downloading package wheel from
https://files.pythonhosted.org/packages/83/72/611c121b6bd15479cb62f1a425b2e3372e121b324228df28e64cc28b01c2/wheel-0.35.1.tar.gz...
Traceback (most recent call last):
  File "utils/scanpypi", line 754, in <module>
    main()
  File "utils/scanpypi", line 702, in main
    package.load_setup()
  File "utils/scanpypi", line 302, in load_setup
    setup = imp.load_module('setup', s_file, s_path, s_desc)
  File "/usr/lib/python3.8/imp.py", line 234, in load_module
    return load_source(name, filename, file)
  File "/usr/lib/python3.8/imp.py", line 171, in load_source
    module = _load(spec)
  File "<frozen importlib._bootstrap>", line 702, in _load
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 783, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/tmp/scanpypi-evf4l461/python-wheel/wheel-0.35.1/setup.py", line 4, in
<module>
  File "utils/scanpypi", line 63, in closure
    BuildrootPackage.setup_args[kwargs['name']] = kwargs
KeyError: 'name'


To reproduce just type: 

utils/scanpypi wheel

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Buildroot] [Bug 13321] utils/scanpypi: fails with packages that define a package’s metadata in setup.cfg

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=13321

--- Comment #1 from kirby at kjn.com.au ---
I'm seeing something similar trying to make a package for python-debugpy. 
I have yet to find a workaround unless anyone has any suggestions?

user at ubuntu:~/buildroot-2020.05.1$ python3 utils/scanpypi debugpy -o package
utils/scanpypi:23: DeprecationWarning: the imp module is deprecated in favour
of importlib; see the module's documentation for alternative uses
  import imp
spdx_lookup module is not installed. This can lead to an inaccurate licence
detection. Please install it via
pip install spdx_lookup
buildroot package name for debugpy: python-debugpy
Package: python-debugpy
Fetching package debugpy
Downloading package debugpy from
https://files.pythonhosted.org/packages/12/73/35e48dfeb1e26fbc6fe6e888c5b7507f7010488bb1d7363322edc38c03c6/debugpy-1.2.0.zip...
Error: Could not install package debugpy: module 'setup' has no attribute
'main'

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Buildroot] [Bug 13321] utils/scanpypi: fails with packages that define a package’s metadata in setup.cfg

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=13321

--- Comment #2 from Oliwia Zaremba <g.oliwia@gmail.com> ---
To provide more background on this issue: this is the result of PEP-517
(https://www.python.org/dev/peps/pep-0517/) getting a wider adoption. According
to the documentation:
https://setuptools.readthedocs.io/en/latest/setuptools.html#id4 the recommended
approach for Python packages is to define the dependencies decoratively in
setup.cfg. The deprecated calls to setup.py remain backwards compatible by
emulating a dummy setup.py with a setup() method call. In that case the method
is called without any arguments, and this is ultimately the reason
utils/scanpypi script fails for many packages that use this new approach. 

It seems to me that one possible solution to this would be to detect if the
setup.cfg exists and retrieve the package's metadata from it (e.g. using the
read_configuration method from setuptools.config). But I wonder if there exists
a Python native solution to retrieve the metadata (including install_requires
dependencies) from a Python package regardless whether PEP-517 convention is
used or not, and more importantly, without installing that package. Does anyone
know of such mechanism?

Also if anyone found a working solution to this, I would very much appreciate
if it can be shared.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Buildroot] [Bug 13321] utils/scanpypi: fails with packages that define a package’s metadata in setup.cfg

[bugzilla]

https://bugs.busybox.net/show_bug.cgi?id=13321

julian@meter.com changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |julian@meter.com

Yann E. MORIN <yann.morin.1998@free.fr> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |kr886q@att.com

--- Comment #3 from Yann E. MORIN <yann.morin.1998@free.fr> ---
*** Bug 14321 has been marked as a duplicate of this bug. ***

-- 
You are receiving this mail because:
You are on the CC list for the bug.
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot