[Qemu-devel] [RFC] Require Python 3 for building QEMU

[Qemu-devel] [RFC] Require Python 3 for building QEMU

[Eduardo Habkost]

Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
---
I'd like to do this in QEMU 3.1.  I think it's time to drop
support for old systems that have only Python 2.

We still have a few scripts that are not required for building
QEMU that still work only with Python 2 (iotests being the most
relevant set).  Requiring Python 3 for building QEMU won't
prevent people from using those scripts with Python 2 until they
are finally ported.
---
 configure   | 8 ++++----
 .travis.yml | 2 +-
 2 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/configure b/configure
index f89d293585..43b45745dc 100755
--- a/configure
+++ b/configure
@@ -885,7 +885,7 @@ fi
 
 : ${make=${MAKE-make}}
 : ${install=${INSTALL-install}}
-: ${python=${PYTHON-python}}
+: ${python=${PYTHON-python3}}
 : ${smbd=${SMBD-/usr/sbin/smbd}}
 
 # Default objcc to clang if available, otherwise use CC
@@ -1730,13 +1730,13 @@ exit 0
 fi
 
 if ! has $python; then
-  error_exit "Python not found. Use --python=/path/to/python"
+  error_exit "Python 3 not found. Use --python=/path/to/python"
 fi
 
 # Note that if the Python conditional here evaluates True we will exit
 # with status 1 which is a shell 'false' value.
-if ! $python -c 'import sys; sys.exit(sys.version_info < (2,7))'; then
-  error_exit "Cannot use '$python', Python 2 >= 2.7 or Python 3 is required." \
+if ! $python -c 'import sys; sys.exit(sys.version_info < (3,0))'; then
+  error_exit "Cannot use '$python', Python 3 is required." \
       "Use --python=/path/to/python to specify a supported Python."
 fi
 
diff --git a/.travis.yml b/.travis.yml
index 95be6ec59f..caca9685fa 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -5,7 +5,7 @@ sudo: false
 dist: trusty
 language: c
 python:
-  - "2.6"
+  - "3.6"
 compiler:
   - gcc
 cache: ccache
-- 
2.18.0.rc1.1.g3f1ff2140

Re: [Qemu-devel] [RFC] Require Python 3 for building QEMU

[Eric Blake]

On 10/13/18 12:02 AM, Eduardo Habkost wrote:
> Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
> ---

Commit message is sparse; can you do something like commit e7b3af81 
where you call out the availability of python 3 on all the platforms 
that are currently supported per our platforms doc?

> I'd like to do this in QEMU 3.1.  I think it's time to drop
> support for old systems that have only Python 2.
> 
> We still have a few scripts that are not required for building
> QEMU that still work only with Python 2 (iotests being the most
> relevant set).  Requiring Python 3 for building QEMU won't
> prevent people from using those scripts with Python 2 until they
> are finally ported.

How much effort is it to complete the porting?

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

Re: [Qemu-devel] [RFC] Require Python 3 for building QEMU

[Max Reitz]

On 13.10.18 07:02, Eduardo Habkost wrote:
> Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
> ---
> I'd like to do this in QEMU 3.1.  I think it's time to drop
> support for old systems that have only Python 2.
> 
> We still have a few scripts that are not required for building
> QEMU that still work only with Python 2 (iotests being the most
> relevant set).  Requiring Python 3 for building QEMU won't
> prevent people from using those scripts with Python 2 until they
> are finally ported.

It very much does because the iotests specifically use the python path
qemu was configured with.

To fix this, configure would need to write something else for into
tests/qemu-iotests/common.env for $PYTHON.  But what?  I don't really
want to introduce a new configure option for this.

So the real fix is indeed to make the iotests work with Python 3, and I
think that needs to be done before we can require Python 3.  Maybe it
even needs to be done at the same time.

Max

> ---
>  configure   | 8 ++++----
>  .travis.yml | 2 +-
>  2 files changed, 5 insertions(+), 5 deletions(-)
> 
> diff --git a/configure b/configure
> index f89d293585..43b45745dc 100755
> --- a/configure
> +++ b/configure
> @@ -885,7 +885,7 @@ fi
>  
>  : ${make=${MAKE-make}}
>  : ${install=${INSTALL-install}}
> -: ${python=${PYTHON-python}}
> +: ${python=${PYTHON-python3}}
>  : ${smbd=${SMBD-/usr/sbin/smbd}}
>  
>  # Default objcc to clang if available, otherwise use CC
> @@ -1730,13 +1730,13 @@ exit 0
>  fi
>  
>  if ! has $python; then
> -  error_exit "Python not found. Use --python=/path/to/python"
> +  error_exit "Python 3 not found. Use --python=/path/to/python"
>  fi
>  
>  # Note that if the Python conditional here evaluates True we will exit
>  # with status 1 which is a shell 'false' value.
> -if ! $python -c 'import sys; sys.exit(sys.version_info < (2,7))'; then
> -  error_exit "Cannot use '$python', Python 2 >= 2.7 or Python 3 is required." \
> +if ! $python -c 'import sys; sys.exit(sys.version_info < (3,0))'; then
> +  error_exit "Cannot use '$python', Python 3 is required." \
>        "Use --python=/path/to/python to specify a supported Python."
>  fi
>  
> diff --git a/.travis.yml b/.travis.yml
> index 95be6ec59f..caca9685fa 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -5,7 +5,7 @@ sudo: false
>  dist: trusty
>  language: c
>  python:
> -  - "2.6"
> +  - "3.6"
>  compiler:
>    - gcc
>  cache: ccache
> 

Re: [Qemu-devel] [RFC] Require Python 3 for building QEMU

[Eduardo Habkost]

On Sat, Oct 13, 2018 at 08:20:25PM +0200, Max Reitz wrote:
> On 13.10.18 07:02, Eduardo Habkost wrote:
> > Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
> > ---
> > I'd like to do this in QEMU 3.1.  I think it's time to drop
> > support for old systems that have only Python 2.
> > 
> > We still have a few scripts that are not required for building
> > QEMU that still work only with Python 2 (iotests being the most
> > relevant set).  Requiring Python 3 for building QEMU won't
> > prevent people from using those scripts with Python 2 until they
> > are finally ported.
> 
> It very much does because the iotests specifically use the python path
> qemu was configured with.
> 
> To fix this, configure would need to write something else for into
> tests/qemu-iotests/common.env for $PYTHON.  But what?  I don't really
> want to introduce a new configure option for this.

What's wrong with '/usr/bin/env python2' and just using the
python2 binary from $PATH?  Why do we need to make the Python
interpreter path for iotests configurable?

> 
> So the real fix is indeed to make the iotests work with Python 3, and I
> think that needs to be done before we can require Python 3.  Maybe it
> even needs to be done at the same time.

I agree that this would be even better.  I just don't think the
pending iotest porting should force all the rest of the build
scripts to be compatible with Python 2.

-- 
Eduardo

Re: [Qemu-devel] [RFC] Require Python 3 for building QEMU

[Max Reitz]

On 13.10.18 22:36, Eduardo Habkost wrote:
> On Sat, Oct 13, 2018 at 08:20:25PM +0200, Max Reitz wrote:
>> On 13.10.18 07:02, Eduardo Habkost wrote:
>>> Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
>>> ---
>>> I'd like to do this in QEMU 3.1.  I think it's time to drop
>>> support for old systems that have only Python 2.
>>>
>>> We still have a few scripts that are not required for building
>>> QEMU that still work only with Python 2 (iotests being the most
>>> relevant set).  Requiring Python 3 for building QEMU won't
>>> prevent people from using those scripts with Python 2 until they
>>> are finally ported.
>>
>> It very much does because the iotests specifically use the python path
>> qemu was configured with.
>>
>> To fix this, configure would need to write something else for into
>> tests/qemu-iotests/common.env for $PYTHON.  But what?  I don't really
>> want to introduce a new configure option for this.
> 
> What's wrong with '/usr/bin/env python2' and just using the
> python2 binary from $PATH?  Why do we need to make the Python
> interpreter path for iotests configurable?

Nothing, sounds good.  No idea why I discarded that idea.

>> So the real fix is indeed to make the iotests work with Python 3, and I
>> think that needs to be done before we can require Python 3.  Maybe it
>> even needs to be done at the same time.
> 
> I agree that this would be even better.  I just don't think the
> pending iotest porting should force all the rest of the build
> scripts to be compatible with Python 2.

True.  It just means that we have to do something about the iotests
before this patch can be merged.

Max

Re: [Qemu-devel] [RFC] Require Python 3 for building QEMU

[Markus Armbruster]

Max Reitz <mreitz@redhat.com> writes:

> On 13.10.18 22:36, Eduardo Habkost wrote:
>> On Sat, Oct 13, 2018 at 08:20:25PM +0200, Max Reitz wrote:
>>> On 13.10.18 07:02, Eduardo Habkost wrote:
>>>> Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
>>>> ---
>>>> I'd like to do this in QEMU 3.1.  I think it's time to drop
>>>> support for old systems that have only Python 2.
>>>>
>>>> We still have a few scripts that are not required for building
>>>> QEMU that still work only with Python 2 (iotests being the most
>>>> relevant set).  Requiring Python 3 for building QEMU won't
>>>> prevent people from using those scripts with Python 2 until they
>>>> are finally ported.
>>>
>>> It very much does because the iotests specifically use the python path
>>> qemu was configured with.
>>>
>>> To fix this, configure would need to write something else for into
>>> tests/qemu-iotests/common.env for $PYTHON.  But what?  I don't really
>>> want to introduce a new configure option for this.
>> 
>> What's wrong with '/usr/bin/env python2' and just using the
>> python2 binary from $PATH?  Why do we need to make the Python
>> interpreter path for iotests configurable?
>
> Nothing, sounds good.  No idea why I discarded that idea.
>
>>> So the real fix is indeed to make the iotests work with Python 3, and I
>>> think that needs to be done before we can require Python 3.  Maybe it
>>> even needs to be done at the same time.
>> 
>> I agree that this would be even better.  I just don't think the
>> pending iotest porting should force all the rest of the build
>> scripts to be compatible with Python 2.
>
> True.  It just means that we have to do something about the iotests
> before this patch can be merged.

I keep hearing about that "we" guy, and all the stuff he has to do, but
I've never seen him deliver anything.

Re: [Qemu-devel] [RFC] Require Python 3 for building QEMU

[Max Reitz]

On 15.10.18 07:55, Markus Armbruster wrote:
> Max Reitz <mreitz@redhat.com> writes:
> 
>> On 13.10.18 22:36, Eduardo Habkost wrote:
>>> On Sat, Oct 13, 2018 at 08:20:25PM +0200, Max Reitz wrote:
>>>> On 13.10.18 07:02, Eduardo Habkost wrote:
>>>>> Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
>>>>> ---
>>>>> I'd like to do this in QEMU 3.1.  I think it's time to drop
>>>>> support for old systems that have only Python 2.
>>>>>
>>>>> We still have a few scripts that are not required for building
>>>>> QEMU that still work only with Python 2 (iotests being the most
>>>>> relevant set).  Requiring Python 3 for building QEMU won't
>>>>> prevent people from using those scripts with Python 2 until they
>>>>> are finally ported.
>>>>
>>>> It very much does because the iotests specifically use the python path
>>>> qemu was configured with.
>>>>
>>>> To fix this, configure would need to write something else for into
>>>> tests/qemu-iotests/common.env for $PYTHON.  But what?  I don't really
>>>> want to introduce a new configure option for this.
>>>
>>> What's wrong with '/usr/bin/env python2' and just using the
>>> python2 binary from $PATH?  Why do we need to make the Python
>>> interpreter path for iotests configurable?
>>
>> Nothing, sounds good.  No idea why I discarded that idea.
>>
>>>> So the real fix is indeed to make the iotests work with Python 3, and I
>>>> think that needs to be done before we can require Python 3.  Maybe it
>>>> even needs to be done at the same time.
>>>
>>> I agree that this would be even better.  I just don't think the
>>> pending iotest porting should force all the rest of the build
>>> scripts to be compatible with Python 2.
>>
>> True.  It just means that we have to do something about the iotests
>> before this patch can be merged.
> 
> I keep hearing about that "we" guy, and all the stuff he has to do, but
> I've never seen him deliver anything.

I've actually been working on it since yesterday.

Max

Re: [Qemu-devel] [RFC] Require Python 3 for building QEMU

[Max Reitz]

On 15.10.18 13:01, Max Reitz wrote:
> On 15.10.18 07:55, Markus Armbruster wrote:
>> Max Reitz <mreitz@redhat.com> writes:
>>
>>> On 13.10.18 22:36, Eduardo Habkost wrote:
>>>> On Sat, Oct 13, 2018 at 08:20:25PM +0200, Max Reitz wrote:
>>>>> On 13.10.18 07:02, Eduardo Habkost wrote:
>>>>>> Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
>>>>>> ---
>>>>>> I'd like to do this in QEMU 3.1.  I think it's time to drop
>>>>>> support for old systems that have only Python 2.
>>>>>>
>>>>>> We still have a few scripts that are not required for building
>>>>>> QEMU that still work only with Python 2 (iotests being the most
>>>>>> relevant set).  Requiring Python 3 for building QEMU won't
>>>>>> prevent people from using those scripts with Python 2 until they
>>>>>> are finally ported.
>>>>>
>>>>> It very much does because the iotests specifically use the python path
>>>>> qemu was configured with.
>>>>>
>>>>> To fix this, configure would need to write something else for into
>>>>> tests/qemu-iotests/common.env for $PYTHON.  But what?  I don't really
>>>>> want to introduce a new configure option for this.
>>>>
>>>> What's wrong with '/usr/bin/env python2' and just using the
>>>> python2 binary from $PATH?  Why do we need to make the Python
>>>> interpreter path for iotests configurable?
>>>
>>> Nothing, sounds good.  No idea why I discarded that idea.
>>>
>>>>> So the real fix is indeed to make the iotests work with Python 3, and I
>>>>> think that needs to be done before we can require Python 3.  Maybe it
>>>>> even needs to be done at the same time.
>>>>
>>>> I agree that this would be even better.  I just don't think the
>>>> pending iotest porting should force all the rest of the build
>>>> scripts to be compatible with Python 2.
>>>
>>> True.  It just means that we have to do something about the iotests
>>> before this patch can be merged.
>>
>> I keep hearing about that "we" guy, and all the stuff he has to do, but
>> I've never seen him deliver anything.
> 
> I've actually been working on it since yesterday.

Also, nobody has to do anything.  It's just that before this very patch
proposed here can go in, someone needs to do something about the
iotests.  If nobody does anything, this patch can't go in as-is.  Simple
as that.

Max

Re: [Qemu-devel] [RFC] Require Python 3 for building QEMU

[Daniel P. Berrangé]

On Sat, Oct 13, 2018 at 02:02:27AM -0300, Eduardo Habkost wrote:
> Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
> ---
> I'd like to do this in QEMU 3.1.  I think it's time to drop
> support for old systems that have only Python 2.
>
> We still have a few scripts that are not required for building
> QEMU that still work only with Python 2 (iotests being the most
> relevant set).  Requiring Python 3 for building QEMU won't
> prevent people from using those scripts with Python 2 until they
> are finally ported.

I think it is premature & unecessary to do this. We just got QEMU building
with dual Python2/3 in 3.0 to give people leeway in the migration path to
a fully v3 future. The code to support building 2/3 in parallel is not
imposing a unreasonable maint burden. Dropping py2 suport would have
negligible impact on the code, as there's no v3-only features we have
used. IOW, I don't think there's a compelling reason to rush into forcing
users onto v3.

If we want to drop py2, we should give people a warning of such a planned
change, especially since some of our targetted host OS[1] don't even
include a py3 as standard without acquiring extra add-on repos. Devs in
a typical corporate env will not have the freedom to install such extra
repos on their machines.

Regards,
Daniel

[1] https://qemu.weilnetz.de/doc/qemu-doc.html#Supported-build-platforms
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [Qemu-devel] [RFC] Require Python 3 for building QEMU

[Peter Maydell]

On 15 October 2018 at 10:32, Daniel P. Berrangé <berrange@redhat.com> wrote:
> On Sat, Oct 13, 2018 at 02:02:27AM -0300, Eduardo Habkost wrote:
>> Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
>> ---
>> I'd like to do this in QEMU 3.1.  I think it's time to drop
>> support for old systems that have only Python 2.
>>
>> We still have a few scripts that are not required for building
>> QEMU that still work only with Python 2 (iotests being the most
>> relevant set).  Requiring Python 3 for building QEMU won't
>> prevent people from using those scripts with Python 2 until they
>> are finally ported.
>
> I think it is premature & unecessary to do this. We just got QEMU building
> with dual Python2/3 in 3.0 to give people leeway in the migration path to
> a fully v3 future. The code to support building 2/3 in parallel is not
> imposing a unreasonable maint burden. Dropping py2 suport would have
> negligible impact on the code, as there's no v3-only features we have
> used. IOW, I don't think there's a compelling reason to rush into forcing
> users onto v3.
>
> If we want to drop py2, we should give people a warning of such a planned
> change, especially since some of our targetted host OS[1] don't even
> include a py3 as standard without acquiring extra add-on repos. Devs in
> a typical corporate env will not have the freedom to install such extra
> repos on their machines.

I agree. I also think that dropping python 2 support before we've
even converted all our python scripts to handle python 3 is the
wrong order to do things. People interested in moving forward with
the transition to python-3-only should start by making sure everything
we have works with python 3...

thanks
-- PMM

Re: [Qemu-devel] [RFC] Require Python 3 for building QEMU

[Daniel P. Berrangé]

On Mon, Oct 15, 2018 at 11:02:03AM +0100, Peter Maydell wrote:
> On 15 October 2018 at 10:32, Daniel P. Berrangé <berrange@redhat.com> wrote:
> > On Sat, Oct 13, 2018 at 02:02:27AM -0300, Eduardo Habkost wrote:
> >> Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
> >> ---
> >> I'd like to do this in QEMU 3.1.  I think it's time to drop
> >> support for old systems that have only Python 2.
> >>
> >> We still have a few scripts that are not required for building
> >> QEMU that still work only with Python 2 (iotests being the most
> >> relevant set).  Requiring Python 3 for building QEMU won't
> >> prevent people from using those scripts with Python 2 until they
> >> are finally ported.
> >
> > I think it is premature & unecessary to do this. We just got QEMU building
> > with dual Python2/3 in 3.0 to give people leeway in the migration path to
> > a fully v3 future. The code to support building 2/3 in parallel is not
> > imposing a unreasonable maint burden. Dropping py2 suport would have
> > negligible impact on the code, as there's no v3-only features we have
> > used. IOW, I don't think there's a compelling reason to rush into forcing
> > users onto v3.
> >
> > If we want to drop py2, we should give people a warning of such a planned
> > change, especially since some of our targetted host OS[1] don't even
> > include a py3 as standard without acquiring extra add-on repos. Devs in
> > a typical corporate env will not have the freedom to install such extra
> > repos on their machines.
> 
> I agree. I also think that dropping python 2 support before we've
> even converted all our python scripts to handle python 3 is the
> wrong order to do things. People interested in moving forward with
> the transition to python-3-only should start by making sure everything
> we have works with python 3...

FWIW,  I would /not/ object to the iotests directory being converted to
pure py3 only, skipping 2/3 compat, if that made it easier, since it is
not a commonly used thing by most people building QEMU.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [Qemu-devel] [RFC] Require Python 3 for building QEMU

[Kevin Wolf]

Am 15.10.2018 um 12:02 hat Peter Maydell geschrieben:
> On 15 October 2018 at 10:32, Daniel P. Berrangé <berrange@redhat.com> wrote:
> > On Sat, Oct 13, 2018 at 02:02:27AM -0300, Eduardo Habkost wrote:
> >> Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
> >> ---
> >> I'd like to do this in QEMU 3.1.  I think it's time to drop
> >> support for old systems that have only Python 2.
> >>
> >> We still have a few scripts that are not required for building
> >> QEMU that still work only with Python 2 (iotests being the most
> >> relevant set).  Requiring Python 3 for building QEMU won't
> >> prevent people from using those scripts with Python 2 until they
> >> are finally ported.
> >
> > I think it is premature & unecessary to do this. We just got QEMU building
> > with dual Python2/3 in 3.0 to give people leeway in the migration path to
> > a fully v3 future. The code to support building 2/3 in parallel is not
> > imposing a unreasonable maint burden. Dropping py2 suport would have
> > negligible impact on the code, as there's no v3-only features we have
> > used. IOW, I don't think there's a compelling reason to rush into forcing
> > users onto v3.
> >
> > If we want to drop py2, we should give people a warning of such a planned
> > change, especially since some of our targetted host OS[1] don't even
> > include a py3 as standard without acquiring extra add-on repos. Devs in
> > a typical corporate env will not have the freedom to install such extra
> > repos on their machines.
> 
> I agree. I also think that dropping python 2 support before we've
> even converted all our python scripts to handle python 3 is the
> wrong order to do things. People interested in moving forward with
> the transition to python-3-only should start by making sure everything
> we have works with python 3...

It's easier to port stuff to Python 3 though than making them work with
both. I think Eduardo's RFC is in part motivated by a patch from
Philippe that converted something in iotests to work with Python 3,
passed review and then turned out to break Python 2.

Having to test every iotests patch twice with different Python versions
isn't something I would like to do for extended periods of time.

Kevin

Re: [Qemu-devel] [RFC] Require Python 3 for building QEMU

[Markus Armbruster]

Kevin Wolf <kwolf@redhat.com> writes:

> Am 15.10.2018 um 12:02 hat Peter Maydell geschrieben:
>> On 15 October 2018 at 10:32, Daniel P. Berrangé <berrange@redhat.com> wrote:
>> > On Sat, Oct 13, 2018 at 02:02:27AM -0300, Eduardo Habkost wrote:
>> >> Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
>> >> ---
>> >> I'd like to do this in QEMU 3.1.  I think it's time to drop
>> >> support for old systems that have only Python 2.
>> >>
>> >> We still have a few scripts that are not required for building
>> >> QEMU that still work only with Python 2 (iotests being the most
>> >> relevant set).  Requiring Python 3 for building QEMU won't
>> >> prevent people from using those scripts with Python 2 until they
>> >> are finally ported.
>> >
>> > I think it is premature & unecessary to do this. We just got QEMU building
>> > with dual Python2/3 in 3.0 to give people leeway in the migration path to
>> > a fully v3 future. The code to support building 2/3 in parallel is not
>> > imposing a unreasonable maint burden. Dropping py2 suport would have
>> > negligible impact on the code, as there's no v3-only features we have
>> > used. IOW, I don't think there's a compelling reason to rush into forcing
>> > users onto v3.
>> >
>> > If we want to drop py2, we should give people a warning of such a planned
>> > change, especially since some of our targetted host OS[1] don't even
>> > include a py3 as standard without acquiring extra add-on repos. Devs in
>> > a typical corporate env will not have the freedom to install such extra
>> > repos on their machines.
>> 
>> I agree. I also think that dropping python 2 support before we've
>> even converted all our python scripts to handle python 3 is the
>> wrong order to do things. People interested in moving forward with
>> the transition to python-3-only should start by making sure everything
>> we have works with python 3...
>
> It's easier to port stuff to Python 3 though than making them work with
> both. I think Eduardo's RFC is in part motivated by a patch from
> Philippe that converted something in iotests to work with Python 3,
> passed review and then turned out to break Python 2.

Seconded.  This is not about the cost of maintaining existing
compatibility gunk, it's about the extra effort to first get the
remainder to work with 2 and 3, only to throw away 2 a few months later.

I propose we permit ourselves to port stuff that isn't essential to
building QEMU straight to 3 instead.  This includes iotests.

> Having to test every iotests patch twice with different Python versions
> isn't something I would like to do for extended periods of time.

It's worth doing only if the benefits of doing it outweigh the costs.  I
don't think they do.

Re: [Qemu-devel] [RFC] Require Python 3 for building QEMU

[Peter Maydell]

On 15 October 2018 at 17:33, Markus Armbruster <armbru@redhat.com> wrote:
> Kevin Wolf <kwolf@redhat.com> writes:
>
>> Am 15.10.2018 um 12:02 hat Peter Maydell geschrieben:
>> It's easier to port stuff to Python 3 though than making them work with
>> both. I think Eduardo's RFC is in part motivated by a patch from
>> Philippe that converted something in iotests to work with Python 3,
>> passed review and then turned out to break Python 2.
>
> Seconded.  This is not about the cost of maintaining existing
> compatibility gunk, it's about the extra effort to first get the
> remainder to work with 2 and 3, only to throw away 2 a few months later.
>
> I propose we permit ourselves to port stuff that isn't essential to
> building QEMU straight to 3 instead.  This includes iotests.

No particular objection, as long as nothing run via 'make'
or 'make check' needs Python 3.

I also suspect "a few months" is an underestimate. My guess
would be we're going to want to keep Python 2 support for
at least the next year, maybe two.

thanks
-- PMM

Re: [Qemu-devel] [RFC] Require Python 3 for building QEMU

[Eduardo Habkost]

On Mon, Oct 15, 2018 at 05:55:27PM +0100, Peter Maydell wrote:
> On 15 October 2018 at 17:33, Markus Armbruster <armbru@redhat.com> wrote:
> > Kevin Wolf <kwolf@redhat.com> writes:
> >
> >> Am 15.10.2018 um 12:02 hat Peter Maydell geschrieben:
> >> It's easier to port stuff to Python 3 though than making them work with
> >> both. I think Eduardo's RFC is in part motivated by a patch from
> >> Philippe that converted something in iotests to work with Python 3,
> >> passed review and then turned out to break Python 2.
> >
> > Seconded.  This is not about the cost of maintaining existing
> > compatibility gunk, it's about the extra effort to first get the
> > remainder to work with 2 and 3, only to throw away 2 a few months later.
> >
> > I propose we permit ourselves to port stuff that isn't essential to
> > building QEMU straight to 3 instead.  This includes iotests.
> 
> No particular objection, as long as nothing run via 'make'
> or 'make check' needs Python 3.

Sounds like a good plan for 3.1.

But:

> 
> I also suspect "a few months" is an underestimate. My guess
> would be we're going to want to keep Python 2 support for
> at least the next year, maybe two.

Python 2.7 will die in less than 15 months[1].  I really want us
to stop reviewing and maintaining Python 2 code in QEMU in less
than 1 year.  Preferably in less than 6 months.

[1] https://pythonclock.org/

-- 
Eduardo

Re: [Qemu-devel] [RFC] Require Python 3 for building QEMU

[Markus Armbruster]

Eduardo Habkost <ehabkost@redhat.com> writes:

> On Mon, Oct 15, 2018 at 05:55:27PM +0100, Peter Maydell wrote:
>> On 15 October 2018 at 17:33, Markus Armbruster <armbru@redhat.com> wrote:
>> > Kevin Wolf <kwolf@redhat.com> writes:
>> >
>> >> Am 15.10.2018 um 12:02 hat Peter Maydell geschrieben:
>> >> It's easier to port stuff to Python 3 though than making them work with
>> >> both. I think Eduardo's RFC is in part motivated by a patch from
>> >> Philippe that converted something in iotests to work with Python 3,
>> >> passed review and then turned out to break Python 2.
>> >
>> > Seconded.  This is not about the cost of maintaining existing
>> > compatibility gunk, it's about the extra effort to first get the
>> > remainder to work with 2 and 3, only to throw away 2 a few months later.
>> >
>> > I propose we permit ourselves to port stuff that isn't essential to
>> > building QEMU straight to 3 instead.  This includes iotests.
>> 
>> No particular objection, as long as nothing run via 'make'
>> or 'make check' needs Python 3.
>
> Sounds like a good plan for 3.1.
>
> But:
>
>> 
>> I also suspect "a few months" is an underestimate. My guess
>> would be we're going to want to keep Python 2 support for
>> at least the next year, maybe two.
>
> Python 2.7 will die in less than 15 months[1].  I really want us
> to stop reviewing and maintaining Python 2 code in QEMU in less
> than 1 year.  Preferably in less than 6 months.

Seconded.

> [1] https://pythonclock.org/

1 year, 2 months, 15 days, 17 hours, 1 minute, 36 seconds, and counting.

Re: [Qemu-devel] [RFC] Require Python 3 for building QEMU

[Peter Maydell]

On 15 October 2018 at 19:01, Eduardo Habkost <ehabkost@redhat.com> wrote:
> On Mon, Oct 15, 2018 at 05:55:27PM +0100, Peter Maydell wrote:
>> I also suspect "a few months" is an underestimate. My guess
>> would be we're going to want to keep Python 2 support for
>> at least the next year, maybe two.
>
> Python 2.7 will die in less than 15 months[1].  I really want us
> to stop reviewing and maintaining Python 2 code in QEMU in less
> than 1 year.  Preferably in less than 6 months.

Well, as with all our dependencies it depends on what the
default python is in the various LTS distro versions we
support. Once the last LTS distro with python 2 falls off
our this-is-what-we-support list we can drop 2 support,
possibly with a deprecate-and-warn-then-drop cycle.

thanks
-- PMM

Re: [Qemu-devel] [RFC] Require Python 3 for building QEMU

[Christian Borntraeger]

On 10/15/2018 06:33 PM, Markus Armbruster wrote:
> Kevin Wolf <kwolf@redhat.com> writes:
> 
>> Am 15.10.2018 um 12:02 hat Peter Maydell geschrieben:
>>> On 15 October 2018 at 10:32, Daniel P. Berrangé <berrange@redhat.com> wrote:
>>>> On Sat, Oct 13, 2018 at 02:02:27AM -0300, Eduardo Habkost wrote:
>>>>> Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
>>>>> ---
>>>>> I'd like to do this in QEMU 3.1.  I think it's time to drop
>>>>> support for old systems that have only Python 2.
>>>>>
>>>>> We still have a few scripts that are not required for building
>>>>> QEMU that still work only with Python 2 (iotests being the most
>>>>> relevant set).  Requiring Python 3 for building QEMU won't
>>>>> prevent people from using those scripts with Python 2 until they
>>>>> are finally ported.
>>>>
>>>> I think it is premature & unecessary to do this. We just got QEMU building
>>>> with dual Python2/3 in 3.0 to give people leeway in the migration path to
>>>> a fully v3 future. The code to support building 2/3 in parallel is not
>>>> imposing a unreasonable maint burden. Dropping py2 suport would have
>>>> negligible impact on the code, as there's no v3-only features we have
>>>> used. IOW, I don't think there's a compelling reason to rush into forcing
>>>> users onto v3.
>>>>
>>>> If we want to drop py2, we should give people a warning of such a planned
>>>> change, especially since some of our targetted host OS[1] don't even
>>>> include a py3 as standard without acquiring extra add-on repos. Devs in
>>>> a typical corporate env will not have the freedom to install such extra
>>>> repos on their machines.
>>>
>>> I agree. I also think that dropping python 2 support before we've
>>> even converted all our python scripts to handle python 3 is the
>>> wrong order to do things. People interested in moving forward with
>>> the transition to python-3-only should start by making sure everything
>>> we have works with python 3...
>>
>> It's easier to port stuff to Python 3 though than making them work with
>> both. I think Eduardo's RFC is in part motivated by a patch from
>> Philippe that converted something in iotests to work with Python 3,
>> passed review and then turned out to break Python 2.
> 
> Seconded.  This is not about the cost of maintaining existing
> compatibility gunk, it's about the extra effort to first get the
> remainder to work with 2 and 3, only to throw away 2 a few months later.
> 
> I propose we permit ourselves to port stuff that isn't essential to
> building QEMU straight to 3 instead.  This includes iotests.
> 
>> Having to test every iotests patch twice with different Python versions
>> isn't something I would like to do for extended periods of time.
> 
> It's worth doing only if the benefits of doing it outweigh the costs.  I
> don't think they do.

FWIW, I do not care about python 2 vs 3. I just want to emphasize that I 
consider the qemu iotest a very valuable part of the qemu test suite as it
has detected a lot of regressions over the past years. So as long as we keep
that running I am fine.
> 

Re: [Qemu-devel] [RFC] Require Python 3 for building QEMU

[Eduardo Habkost]

On Mon, Oct 15, 2018 at 08:19:18PM +0200, Christian Borntraeger wrote:
> 
> 
> On 10/15/2018 06:33 PM, Markus Armbruster wrote:
> > Kevin Wolf <kwolf@redhat.com> writes:
> > 
> >> Am 15.10.2018 um 12:02 hat Peter Maydell geschrieben:
> >>> On 15 October 2018 at 10:32, Daniel P. Berrangé <berrange@redhat.com> wrote:
> >>>> On Sat, Oct 13, 2018 at 02:02:27AM -0300, Eduardo Habkost wrote:
> >>>>> Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
> >>>>> ---
> >>>>> I'd like to do this in QEMU 3.1.  I think it's time to drop
> >>>>> support for old systems that have only Python 2.
> >>>>>
> >>>>> We still have a few scripts that are not required for building
> >>>>> QEMU that still work only with Python 2 (iotests being the most
> >>>>> relevant set).  Requiring Python 3 for building QEMU won't
> >>>>> prevent people from using those scripts with Python 2 until they
> >>>>> are finally ported.
> >>>>
> >>>> I think it is premature & unecessary to do this. We just got QEMU building
> >>>> with dual Python2/3 in 3.0 to give people leeway in the migration path to
> >>>> a fully v3 future. The code to support building 2/3 in parallel is not
> >>>> imposing a unreasonable maint burden. Dropping py2 suport would have
> >>>> negligible impact on the code, as there's no v3-only features we have
> >>>> used. IOW, I don't think there's a compelling reason to rush into forcing
> >>>> users onto v3.
> >>>>
> >>>> If we want to drop py2, we should give people a warning of such a planned
> >>>> change, especially since some of our targetted host OS[1] don't even
> >>>> include a py3 as standard without acquiring extra add-on repos. Devs in
> >>>> a typical corporate env will not have the freedom to install such extra
> >>>> repos on their machines.
> >>>
> >>> I agree. I also think that dropping python 2 support before we've
> >>> even converted all our python scripts to handle python 3 is the
> >>> wrong order to do things. People interested in moving forward with
> >>> the transition to python-3-only should start by making sure everything
> >>> we have works with python 3...
> >>
> >> It's easier to port stuff to Python 3 though than making them work with
> >> both. I think Eduardo's RFC is in part motivated by a patch from
> >> Philippe that converted something in iotests to work with Python 3,
> >> passed review and then turned out to break Python 2.
> > 
> > Seconded.  This is not about the cost of maintaining existing
> > compatibility gunk, it's about the extra effort to first get the
> > remainder to work with 2 and 3, only to throw away 2 a few months later.
> > 
> > I propose we permit ourselves to port stuff that isn't essential to
> > building QEMU straight to 3 instead.  This includes iotests.
> > 
> >> Having to test every iotests patch twice with different Python versions
> >> isn't something I would like to do for extended periods of time.
> > 
> > It's worth doing only if the benefits of doing it outweigh the costs.  I
> > don't think they do.
> 
> FWIW, I do not care about python 2 vs 3. I just want to emphasize that I 
> consider the qemu iotest a very valuable part of the qemu test suite as it
> has detected a lot of regressions over the past years. So as long as we keep
> that running I am fine.

It depends where exactly one needs to keep them running.  Does
anybody need to run iotests for QEMU 3.0+ builds in systems where
it's impossible to install Python 3?

If somebody really needs that, then I'd kindly ask them to
volunteer to do the extra work to make iotests compatible with
both Python 2 and 3.

-- 
Eduardo

Re: [Qemu-devel] [RFC] Require Python 3 for building QEMU

[Christian Borntraeger]

On 10/15/2018 08:33 PM, Eduardo Habkost wrote:
> On Mon, Oct 15, 2018 at 08:19:18PM +0200, Christian Borntraeger wrote:
[...]

>>>>
>>>> It's easier to port stuff to Python 3 though than making them work with
>>>> both. I think Eduardo's RFC is in part motivated by a patch from
>>>> Philippe that converted something in iotests to work with Python 3,
>>>> passed review and then turned out to break Python 2.
>>>
>>> Seconded.  This is not about the cost of maintaining existing
>>> compatibility gunk, it's about the extra effort to first get the
>>> remainder to work with 2 and 3, only to throw away 2 a few months later.
>>>
>>> I propose we permit ourselves to port stuff that isn't essential to
>>> building QEMU straight to 3 instead.  This includes iotests.
>>>
>>>> Having to test every iotests patch twice with different Python versions
>>>> isn't something I would like to do for extended periods of time.
>>>
>>> It's worth doing only if the benefits of doing it outweigh the costs.  I
>>> don't think they do.
>>
>> FWIW, I do not care about python 2 vs 3. I just want to emphasize that I 
>> consider the qemu iotest a very valuable part of the qemu test suite as it
>> has detected a lot of regressions over the past years. So as long as we keep
>> that running I am fine.
> 
> It depends where exactly one needs to keep them running.  Does
> anybody need to run iotests for QEMU 3.0+ builds in systems where
> it's impossible to install Python 3?

I run that as a regression package. Most distros seems to be fine,
with RHEL7 being the problematic case. But for upstream testing on a redhat
like distro fedora is good enough. 
> 
> If somebody really needs that, then I'd kindly ask them to
> volunteer to do the extra work to make iotests compatible with
> both Python 2 and 3.

As I said. As long as things continue to work on a resonably new distro
things are ok.