Re: [PATCH v6 04/25] python: add qemu package installer

From: Cleber Rosa <crosa@redhat.com>
To: John Snow <jsnow@redhat.com>
Cc: "Kevin Wolf" <kwolf@redhat.com>, "Fam Zheng" <fam@euphon.net>,
	"Thomas Huth" <thuth@redhat.com>,
	"Eduardo Habkost" <ehabkost@redhat.com>,
	qemu-block@nongnu.org,
	"Philippe Mathieu-Daudé" <philmd@redhat.com>,
	qemu-devel@nongnu.org,
	"Wainer dos Santos Moschetta" <wainersm@redhat.com>,
	"Philippe Mathieu-Daudé" <f4bug@amsat.org>,
	"Willian Rampazzo" <willianr@redhat.com>,
	"Willian Rampazzo" <wrampazz@redhat.com>,
	"Max Reitz" <mreitz@redhat.com>,
	"Alex Bennée" <alex.bennee@linaro.org>,
	"Beraldo Leal" <bleal@redhat.com>
Subject: Re: [PATCH v6 04/25] python: add qemu package installer
Date: Fri, 21 May 2021 00:00:01 -0400	[thread overview]
Message-ID: <YKcwI3iA9qTY6Mzc@localhost.localdomain> (raw)
In-Reply-To: <20210512231241.2816122-5-jsnow@redhat.com>

[-- Attachment #1: Type: text/plain, Size: 6467 bytes --]

On Wed, May 12, 2021 at 07:12:20PM -0400, John Snow wrote:
> Add setup.cfg and setup.py, necessary for installing a package via
> pip. Add a ReST document (PACKAGE.rst) explaining the basics of what
> this package is for and who to contact for more information. This
> document will be used as the landing page for the package on PyPI.
> 
> I am not yet using a pyproject.toml style package manifest, because
> "editable" installs are not defined (yet?) by PEP-517/518.
> 
> I consider editable installs crucial for development, though they have
> (apparently) always been somewhat poorly defined.
>

They are crucial for development indeed, so I agree with your overall
choices here.

> Pip now (19.2 and later) now supports editable installs for projects
> using pyproject.toml manifests, but might require the use of the
> --no-use-pep517 flag, which somewhat defeats the point.
>

Just to make it clear for other people reading this, it means that
even with the very latest pip release (21.1.1), you *must* have a
setup.py or setup.cfg file to use editable (development) installs[1]
You can *not* rely solely on a pyproject.toml setup.

[1] https://github.com/pypa/pip/pull/9547/commits/7a95720e796a5e56481c1cc20b6ce6249c50f357

> For now, while the dust settles, stick with the de-facto
> setup.py/setup.cfg combination supported by setuptools. It will be worth
> re-evaluating this point again in the future when our supported build
> platforms all ship a fairly modern pip.
>

Agreed, but let's not hold our breath given that even pip 21.1.1 still
doesn't support that.  My guesstimate is 6-12 months for the feature to
be developed/merged, and let's say another 12 months for our supported
build platforms to ship it.

> Additional reading on this matter:
> 
> https://github.com/pypa/packaging-problems/issues/256
> https://github.com/pypa/pip/issues/6334
> https://github.com/pypa/pip/issues/6375
> https://github.com/pypa/pip/issues/6434
> https://github.com/pypa/pip/issues/6438
> 
> Signed-off-by: John Snow <jsnow@redhat.com>
> ---
>  python/PACKAGE.rst | 33 +++++++++++++++++++++++++++++++++
>  python/setup.cfg   | 19 +++++++++++++++++++
>  python/setup.py    | 23 +++++++++++++++++++++++
>  3 files changed, 75 insertions(+)
>  create mode 100644 python/PACKAGE.rst
>  create mode 100644 python/setup.cfg
>  create mode 100755 python/setup.py
>
> diff --git a/python/PACKAGE.rst b/python/PACKAGE.rst
> new file mode 100644
> index 00000000000..1bbfe1b58e2
> --- /dev/null
> +++ b/python/PACKAGE.rst
> @@ -0,0 +1,33 @@
> +QEMU Python Tooling
> +===================
> +
> +This package provides QEMU tooling used by the QEMU project to build,
> +configure, and test QEMU. It is not a fully-fledged SDK and it is subject
> +to change at any time.
> +
> +Usage
> +-----
> +
> +The ``qemu.qmp`` subpackage provides a library for communicating with
> +QMP servers. The ``qemu.machine`` subpackage offers rudimentary
> +facilities for launching and managing QEMU processes. Refer to each
> +package's documentation
> +(``>>> help(qemu.qmp)``, ``>>> help(qemu.machine)``)
> +for more information.
> +
> +Contributing
> +------------
> +
> +This package is maintained by John Snow <jsnow@redhat.com> as part of
> +the QEMU source tree. Contributions are welcome and follow the `QEMU
> +patch submission process
> +<https://wiki.qemu.org/Contribute/SubmitAPatch>`_, which involves
> +sending patches to the QEMU development mailing list.
> +
> +John maintains a `GitLab staging branch
> +<https://gitlab.com/jsnow/qemu/-/tree/python>`_, and there is an
> +official `GitLab mirror <https://gitlab.com/qemu-project/qemu>`_.
> +
> +Please report bugs on the `QEMU issue tracker
> +<https://gitlab.com/qemu-project/qemu/-/issues>`_ and tag ``@jsnow`` in
> +the report.
> diff --git a/python/setup.cfg b/python/setup.cfg
> new file mode 100644
> index 00000000000..dd71640fc2f
> --- /dev/null
> +++ b/python/setup.cfg
> @@ -0,0 +1,19 @@
> +[metadata]
> +name = qemu
> +maintainer = QEMU Developer Team
> +maintainer_email = qemu-devel@nongnu.org
> +url = https://www.qemu.org/
> +download_url = https://www.qemu.org/download/
> +description = QEMU Python Build, Debug and SDK tooling.
> +long_description = file:PACKAGE.rst
> +long_description_content_type = text/x-rst
> +classifiers =
> +    Development Status :: 3 - Alpha
> +    License :: OSI Approved :: GNU General Public License v2 (GPLv2)
> +    Natural Language :: English
> +    Operating System :: OS Independent
> +    Programming Language :: Python :: 3 :: Only
> +
> +[options]
> +python_requires = >= 3.6
> +packages = find_namespace:
> diff --git a/python/setup.py b/python/setup.py
> new file mode 100755
> index 00000000000..2014f81b757
> --- /dev/null
> +++ b/python/setup.py
> @@ -0,0 +1,23 @@
> +#!/usr/bin/env python3
> +"""
> +QEMU tooling installer script
> +Copyright (c) 2020-2021 John Snow for Red Hat, Inc.
> +"""
> +
> +import setuptools
> +import pkg_resources
> +
> +
> +def main():
> +    """
> +    QEMU tooling installer
> +    """
> +
> +    # https://medium.com/@daveshawley/safely-using-setup-cfg-for-metadata-1babbe54c108
> +    pkg_resources.require('setuptools>=39.2')
> +
> +    setuptools.setup()
> +
> +
> +if __name__ == '__main__':
> +    main()
> -- 
> 2.30.2
> 
> 

BTW, about the need to have a "setup.py", before pip 21.1.1:

  $ rm setup.py
  $ pip install -e .
  ERROR: File "setup.py" not found. Directory cannot be installed in editable mode: /home/cleber/src/qemu/python

On pip 21.1.1:

   $ pip install -e . 
   Obtaining file:///home/cleber/src/qemu/python
   Installing collected packages: qemu
     Running setup.py develop for qemu
   Successfully installed qemu-0.0.0

Side note: The "Running setup.py ..." message given by pip 21.1.1,
even though there is *not* a "setup.py" is rather confusing.

Anyway, we may be able to drop setup.py either when we find pip 21.1.1
or later in our "common build environments", or if we require people
hacking on the Python module to "pip install --upgrade pip".

I'll be repeating myself here, but I believe you made the right
choices at this time, and based on my testing I can successfully
install/develop using "python setup.py" and "pip", so:

Reviewed-by: Cleber Rosa <crosa@redhat.com>
Tested-by: Cleber Rosa <crosa@redhat.com>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]