[Qemu-devel] Deprecation policy and build dependencies

[Qemu-devel] Deprecation policy and build dependencies

[Eduardo Habkost]

Long story short: I would really like to drop support for Python
2 in QEMU 4.1.

What exactly prevents us from doing this?  Does our deprecation
policy really apply to build dependencies?

-- 
Eduardo

Re: [Qemu-devel] Deprecation policy and build dependencies

[John Snow]

On 5/31/19 3:24 PM, Eduardo Habkost wrote:
> Long story short: I would really like to drop support for Python
> 2 in QEMU 4.1.
> 
> What exactly prevents us from doing this?  Does our deprecation
> policy really apply to build dependencies?
> 

Normally I'd say it's only nice to also follow the depreciation policy
for tooling as well to give people a chance to switch away, but with
regards to Python2, I feel like we're in the clear to drop it for the
first release that will happen after the Python2 doomsday clock.

(So, probably 4.2.)

--js

Re: [Qemu-devel] Deprecation policy and build dependencies

[Markus Armbruster]

John Snow <jsnow@redhat.com> writes:

> On 5/31/19 3:24 PM, Eduardo Habkost wrote:
>> Long story short: I would really like to drop support for Python
>> 2 in QEMU 4.1.

The sooner, the better, as far as I'm concerned.

>> What exactly prevents us from doing this?  Does our deprecation
>> policy really apply to build dependencies?
>> 
>
> Normally I'd say it's only nice to also follow the depreciation policy
> for tooling as well to give people a chance to switch away, but with
> regards to Python2, I feel like we're in the clear to drop it for the
> first release that will happen after the Python2 doomsday clock.
>
> (So, probably 4.2.)

In addition to our feature deprecation policity, we have a "Supported
build platforms" policy (commit 45b47130f4b).  The most common holdback
is this one:

    For distributions with long-lifetime releases, the project will aim
    to support the most recent major version at all times. Support for
    the previous major version will be dropped 2 years after the new
    major version is released. For the purposes of identifying supported
    software versions, the project will look at RHEL, Debian, Ubuntu
    LTS, and SLES distros. Other long-lifetime distros will be assumed
    to ship similar software versions.

RHEL-7 has Python 3 only in EPEL.  RHEL-8 came out last month.  Unless
we interpret our policy to include EPEL, this means supporting Python 2
for some 16 months after upstream Python retires it.  My personal
opinion: nuts.

I didn't bother checking Debian, Ubuntu LTS and SLES.

For hosts other than Linux, we're less ambitious.

Re: [Qemu-devel] Deprecation policy and build dependencies

[John Snow]

On 6/3/19 8:26 AM, Markus Armbruster wrote:
> John Snow <jsnow@redhat.com> writes:
> 
>> On 5/31/19 3:24 PM, Eduardo Habkost wrote:
>>> Long story short: I would really like to drop support for Python
>>> 2 in QEMU 4.1.
> 
> The sooner, the better, as far as I'm concerned.
> 
>>> What exactly prevents us from doing this?  Does our deprecation
>>> policy really apply to build dependencies?
>>>
>>
>> Normally I'd say it's only nice to also follow the depreciation policy
>> for tooling as well to give people a chance to switch away, but with
>> regards to Python2, I feel like we're in the clear to drop it for the
>> first release that will happen after the Python2 doomsday clock.
>>
>> (So, probably 4.2.)
> 
> In addition to our feature deprecation policity, we have a "Supported
> build platforms" policy (commit 45b47130f4b).  The most common holdback
> is this one:
> 
>     For distributions with long-lifetime releases, the project will aim
>     to support the most recent major version at all times. Support for
>     the previous major version will be dropped 2 years after the new
>     major version is released. For the purposes of identifying supported
>     software versions, the project will look at RHEL, Debian, Ubuntu
>     LTS, and SLES distros. Other long-lifetime distros will be assumed
>     to ship similar software versions.
> 
> RHEL-7 has Python 3 only in EPEL.  RHEL-8 came out last month.  Unless
> we interpret our policy to include EPEL, this means supporting Python 2
> for some 16 months after upstream Python retires it.  My personal
> opinion: nuts.
> 

I would rather not support Python2 a day after the clock expires.

> I didn't bother checking Debian, Ubuntu LTS and SLES.
> 
> For hosts other than Linux, we're less ambitious.
> 

That policy strikes me as weird, because RHEL7 is not going to be, in
general, using the latest and greatest QEMU. Usually stable versions of
distros stick with the versions of the programs that came out at the time.

What's the benefit of making sure that stable platforms can continue to
run the *newest* QEMU? Is this even a reasonable restriction? If you are
running RHEL7, how many projects do you expect to be able to git clone
and build and have that work with the rest of your legacy/stable
dependencies?

RHEL7 uses a 1.5.3 based version. I don't think it matters if we update
4.2 to be Python3 only, really.

Re: [Qemu-devel] Deprecation policy and build dependencies

[Cornelia Huck]

On Mon, 3 Jun 2019 14:02:16 -0400
John Snow <jsnow@redhat.com> wrote:

> On 6/3/19 8:26 AM, Markus Armbruster wrote:
> > John Snow <jsnow@redhat.com> writes:
> >   
> >> On 5/31/19 3:24 PM, Eduardo Habkost wrote:  
> >>> Long story short: I would really like to drop support for Python
> >>> 2 in QEMU 4.1.  
> > 
> > The sooner, the better, as far as I'm concerned.
> >   
> >>> What exactly prevents us from doing this?  Does our deprecation
> >>> policy really apply to build dependencies?
> >>>  
> >>
> >> Normally I'd say it's only nice to also follow the depreciation policy
> >> for tooling as well to give people a chance to switch away, but with
> >> regards to Python2, I feel like we're in the clear to drop it for the
> >> first release that will happen after the Python2 doomsday clock.
> >>
> >> (So, probably 4.2.)  
> > 
> > In addition to our feature deprecation policity, we have a "Supported
> > build platforms" policy (commit 45b47130f4b).  The most common holdback
> > is this one:
> > 
> >     For distributions with long-lifetime releases, the project will aim
> >     to support the most recent major version at all times. Support for
> >     the previous major version will be dropped 2 years after the new
> >     major version is released. For the purposes of identifying supported
> >     software versions, the project will look at RHEL, Debian, Ubuntu
> >     LTS, and SLES distros. Other long-lifetime distros will be assumed
> >     to ship similar software versions.
> > 
> > RHEL-7 has Python 3 only in EPEL.  RHEL-8 came out last month.  Unless
> > we interpret our policy to include EPEL, this means supporting Python 2
> > for some 16 months after upstream Python retires it.  My personal
> > opinion: nuts.
> >   
> 
> I would rather not support Python2 a day after the clock expires.
> 
> > I didn't bother checking Debian, Ubuntu LTS and SLES.
> > 
> > For hosts other than Linux, we're less ambitious.
> >   
> 
> That policy strikes me as weird, because RHEL7 is not going to be, in
> general, using the latest and greatest QEMU. Usually stable versions of
> distros stick with the versions of the programs that came out at the time.

I think the idea was that folks might actually develop on a 'stable'
distro (in a previous life, I used to complain quite often that
building QEMU on a stable distro broke... it was one of my main
development machines, but not controlled by me).

> 
> What's the benefit of making sure that stable platforms can continue to
> run the *newest* QEMU? Is this even a reasonable restriction? If you are
> running RHEL7, how many projects do you expect to be able to git clone
> and build and have that work with the rest of your legacy/stable
> dependencies?
> 
> RHEL7 uses a 1.5.3 based version. I don't think it matters if we update
> 4.2 to be Python3 only, really.

It depends on how old the distro is and what update policy it
uses... if parts of it are regularly updated, it might actually be
usable. In this case, I think we really need to interpret our policy
to include EPEL, or it is completely nuts.

Re: [Qemu-devel] Deprecation policy and build dependencies

[Peter Maydell]

On Mon, 3 Jun 2019 at 19:02, John Snow <jsnow@redhat.com> wrote:
> That policy strikes me as weird, because RHEL7 is not going to be, in
> general, using the latest and greatest QEMU. Usually stable versions of
> distros stick with the versions of the programs that came out at the time.
>
> What's the benefit of making sure that stable platforms can continue to
> run the *newest* QEMU? Is this even a reasonable restriction? If you are
> running RHEL7, how many projects do you expect to be able to git clone
> and build and have that work with the rest of your legacy/stable
> dependencies?

The benefit is that in general people who want to build QEMU
from source can do so. I don't want us to be the kind of
project that needs latest-and-greatest-foo for everything
to build, because that sort of project is pretty infuriating
to try to work with if you're an occasional contributor.
"Builds on LTS distros" is an easy-to-express way to keep
things from getting out of hand.

Plus a bunch of the build machines we do testing on are
not running bleeding edge distros, and as Connie says plenty
of developers do QEMU development on non-bleeding-edge versions
(my primary dev box run an LTS Ubuntu).

thanks
-- PMM

Re: [Qemu-devel] Deprecation policy and build dependencies

[John Snow]

On 6/3/19 2:17 PM, Peter Maydell wrote:
> On Mon, 3 Jun 2019 at 19:02, John Snow <jsnow@redhat.com> wrote:
>> That policy strikes me as weird, because RHEL7 is not going to be, in
>> general, using the latest and greatest QEMU. Usually stable versions of
>> distros stick with the versions of the programs that came out at the time.
>>
>> What's the benefit of making sure that stable platforms can continue to
>> run the *newest* QEMU? Is this even a reasonable restriction? If you are
>> running RHEL7, how many projects do you expect to be able to git clone
>> and build and have that work with the rest of your legacy/stable
>> dependencies?
> 
> The benefit is that in general people who want to build QEMU
> from source can do so. I don't want us to be the kind of
> project that needs latest-and-greatest-foo for everything
> to build, because that sort of project is pretty infuriating
> to try to work with if you're an occasional contributor.
> "Builds on LTS distros" is an easy-to-express way to keep
> things from getting out of hand.
> 
> Plus a bunch of the build machines we do testing on are
> not running bleeding edge distros, and as Connie says plenty
> of developers do QEMU development on non-bleeding-edge versions
> (my primary dev box run an LTS Ubuntu).
> 

I get it, we don't want to require Python 3.8 because some dev wanted
assignment conditionals -- but we're talking about Python 2 here, which
suffers its EOL by the end of this calendar year.

So do we think it's reasonable to drop support for Python2 for the
release that comes out after Python2's EOL, or do we insist on 2x3
simultaneous support for years more?

--js

Re: [Qemu-devel] Deprecation policy and build dependencies

[Peter Maydell]

On Mon, 3 Jun 2019 at 19:21, John Snow <jsnow@redhat.com> wrote:
> I get it, we don't want to require Python 3.8 because some dev wanted
> assignment conditionals -- but we're talking about Python 2 here, which
> suffers its EOL by the end of this calendar year.
>
> So do we think it's reasonable to drop support for Python2 for the
> release that comes out after Python2's EOL, or do we insist on 2x3
> simultaneous support for years more?

I don't have a strong opinion on Python in particular, but
I think it would be nicer to avoid the "python is a special
snowflake" effect. Would it really be so bad for it to just
be "drop it when it falls off the last LTS distro" like the
rest of our dependencies ?

thanks
-- PMM

Re: [Qemu-devel] Deprecation policy and build dependencies

[John Snow]

On 6/3/19 2:27 PM, Peter Maydell wrote:
> On Mon, 3 Jun 2019 at 19:21, John Snow <jsnow@redhat.com> wrote:
>> I get it, we don't want to require Python 3.8 because some dev wanted
>> assignment conditionals -- but we're talking about Python 2 here, which
>> suffers its EOL by the end of this calendar year.
>>
>> So do we think it's reasonable to drop support for Python2 for the
>> release that comes out after Python2's EOL, or do we insist on 2x3
>> simultaneous support for years more?
> 
> I don't have a strong opinion on Python in particular, but
> I think it would be nicer to avoid the "python is a special
> snowflake" effect. Would it really be so bad for it to just
> be "drop it when it falls off the last LTS distro" like the
> rest of our dependencies ?
> 
> thanks
> -- PMM
> 

When it comes to supporting both 2 and 3 simultaneously yes; it's in my
opinion not trivial to maintain a growing testing and utility
infrastructure that works in versions as disparate as 2.7 and 3.7 from
RHEL7 all the way to Fedora 30. It's not going to get easier, either.
(Especially not after EOL.)

For comparison, at least when we target different versions of compilers,
we are at least targeting the same version of the language...

If you are running a stable or outdated distro and you want to build
bleeding edge QEMU and you cannot somehow find out how to get Python3 on
your machine after the official EOL of python2, I am not sure that
should become the developer's burden. I don't think we are being too
unreasonably avant-garde about adopting technologies that are too new.

--js

Re: [Qemu-devel] Deprecation policy and build dependencies

[Eduardo Habkost]

On Mon, Jun 03, 2019 at 08:16:29PM +0200, Cornelia Huck wrote:
> On Mon, 3 Jun 2019 14:02:16 -0400
> John Snow <jsnow@redhat.com> wrote:
> 
> > On 6/3/19 8:26 AM, Markus Armbruster wrote:
> > > John Snow <jsnow@redhat.com> writes:
> > >   
> > >> On 5/31/19 3:24 PM, Eduardo Habkost wrote:  
> > >>> Long story short: I would really like to drop support for Python
> > >>> 2 in QEMU 4.1.  
> > > 
> > > The sooner, the better, as far as I'm concerned.
> > >   
> > >>> What exactly prevents us from doing this?  Does our deprecation
> > >>> policy really apply to build dependencies?
> > >>>  
> > >>
> > >> Normally I'd say it's only nice to also follow the depreciation policy
> > >> for tooling as well to give people a chance to switch away, but with
> > >> regards to Python2, I feel like we're in the clear to drop it for the
> > >> first release that will happen after the Python2 doomsday clock.
> > >>
> > >> (So, probably 4.2.)  
> > > 
> > > In addition to our feature deprecation policity, we have a "Supported
> > > build platforms" policy (commit 45b47130f4b).  The most common holdback
> > > is this one:
> > > 
> > >     For distributions with long-lifetime releases, the project will aim
> > >     to support the most recent major version at all times. Support for
> > >     the previous major version will be dropped 2 years after the new
> > >     major version is released. For the purposes of identifying supported
> > >     software versions, the project will look at RHEL, Debian, Ubuntu
> > >     LTS, and SLES distros. Other long-lifetime distros will be assumed
> > >     to ship similar software versions.
> > > 
> > > RHEL-7 has Python 3 only in EPEL.  RHEL-8 came out last month.  Unless
> > > we interpret our policy to include EPEL, this means supporting Python 2
> > > for some 16 months after upstream Python retires it.  My personal
> > > opinion: nuts.
> > >   
> > 
> > I would rather not support Python2 a day after the clock expires.

Me neither.

> > 
> > > I didn't bother checking Debian, Ubuntu LTS and SLES.
> > > 
> > > For hosts other than Linux, we're less ambitious.
> > >   
> > 
> > That policy strikes me as weird, because RHEL7 is not going to be, in
> > general, using the latest and greatest QEMU. Usually stable versions of
> > distros stick with the versions of the programs that came out at the time.
> 
> I think the idea was that folks might actually develop on a 'stable'
> distro (in a previous life, I used to complain quite often that
> building QEMU on a stable distro broke... it was one of my main
> development machines, but not controlled by me).

Good point.

> 
> > 
> > What's the benefit of making sure that stable platforms can continue to
> > run the *newest* QEMU? Is this even a reasonable restriction? If you are
> > running RHEL7, how many projects do you expect to be able to git clone
> > and build and have that work with the rest of your legacy/stable
> > dependencies?
> > 
> > RHEL7 uses a 1.5.3 based version. I don't think it matters if we update
> > 4.2 to be Python3 only, really.
> 
> It depends on how old the distro is and what update policy it
> uses... if parts of it are regularly updated, it might actually be
> usable. In this case, I think we really need to interpret our policy
> to include EPEL, or it is completely nuts.

Let's do that, please.  If we simply include EPEL in our policy
we don't need to treat Python as special.

-- 
Eduardo

Re: [Qemu-devel] Deprecation policy and build dependencies

[Gerd Hoffmann]

  Hi,

> >> Normally I'd say it's only nice to also follow the depreciation policy
> >> for tooling as well to give people a chance to switch away, but with
> >> regards to Python2, I feel like we're in the clear to drop it for the
> >> first release that will happen after the Python2 doomsday clock.
> >>
> >> (So, probably 4.2.)

Makes sense to me.

> > RHEL-7 has Python 3 only in EPEL.

That'll change with 7.7 btw.  Beside that I'd say EPEL is good enough.

> What's the benefit of making sure that stable platforms can continue to
> run the *newest* QEMU?

Well, my development workstation runs RHEL-7 ...

cheers,
  Gerd

Re: [Qemu-devel] Deprecation policy and build dependencies

[Markus Armbruster]

Peter Maydell <peter.maydell@linaro.org> writes:

> On Mon, 3 Jun 2019 at 19:21, John Snow <jsnow@redhat.com> wrote:
>> I get it, we don't want to require Python 3.8 because some dev wanted
>> assignment conditionals -- but we're talking about Python 2 here, which
>> suffers its EOL by the end of this calendar year.

"Not because some dev wanted assignment conditionals" is the non-reason.
Let me spell out the reason: supporting Python 2 is expensive for us.
As the amount of Python code grows, it gets more and more expensive.

Is this really time and effort well spent?

>> So do we think it's reasonable to drop support for Python2 for the
>> release that comes out after Python2's EOL, or do we insist on 2x3
>> simultaneous support for years more?
>
> I don't have a strong opinion on Python in particular, but
> I think it would be nicer to avoid the "python is a special
> snowflake" effect.

Lots of things are nice.  Limited resources dictate we can only get some
of them.

>                    Would it really be so bad for it to just
> be "drop it when it falls off the last LTS distro" like the
> rest of our dependencies ?

In my experience maintaining and evolving the QAPI generators,
supporting both Python 2 and 3 is a constant distraction that adds up
over time.  It's all manual; we decided against adopting one of tool
chains made for dealing with this mess.  I'd rather think about how to
solve real user problems like deprecation information or command line
introspection than deal with Python 2 vs. 3 arcana.

Just the other day, I flagged an innocent-looking simplification of some
non-QAPI Python code that changed semantics subtly on Python 2, impact
unknown.  The developer did not know.  The fact that I waste precious
brain capacity on knowing this shit (pardon my French) is a bloody
shame.  Well, some of this shit, because I've screwed it up myself, too.

The sooner we stop the bleeding, the better.

Re: [Qemu-devel] Deprecation policy and build dependencies

[Philippe Mathieu-Daudé]

On 6/3/19 8:16 PM, Cornelia Huck wrote:
> On Mon, 3 Jun 2019 14:02:16 -0400
> John Snow <jsnow@redhat.com> wrote:
[...]
>> I would rather not support Python2 a day after the clock expires.
>>
>>> I didn't bother checking Debian, Ubuntu LTS and SLES.
>>>
>>> For hosts other than Linux, we're less ambitious.
>>>   
>>
>> That policy strikes me as weird, because RHEL7 is not going to be, in
>> general, using the latest and greatest QEMU. Usually stable versions of
>> distros stick with the versions of the programs that came out at the time.
> 
> I think the idea was that folks might actually develop on a 'stable'
> distro (in a previous life, I used to complain quite often that
> building QEMU on a stable distro broke... it was one of my main
> development machines, but not controlled by me).
> 
>>
>> What's the benefit of making sure that stable platforms can continue to
>> run the *newest* QEMU? Is this even a reasonable restriction? If you are
>> running RHEL7, how many projects do you expect to be able to git clone
>> and build and have that work with the rest of your legacy/stable
>> dependencies?
>>
>> RHEL7 uses a 1.5.3 based version. I don't think it matters if we update
>> 4.2 to be Python3 only, really.
> 
> It depends on how old the distro is and what update policy it
> uses... if parts of it are regularly updated, it might actually be
> usable. In this case, I think we really need to interpret our policy
> to include EPEL, or it is completely nuts.

Red Hat supports Docker on RHEL 7 and above.

Docker solved all my problems, as long as a host can install Docker, I
can build on old or edge images. I don't install plenty of dependencies
on my testing hosts, but in trashable docker images. For regular testing
I use image snapshots.

You need a sysadmin (or root access) to install the docker daemon however.

Re: [Qemu-devel] Deprecation policy and build dependencies

[John Snow]

On 6/4/19 1:31 AM, Markus Armbruster wrote:
> Peter Maydell <peter.maydell@linaro.org> writes:
> 
>> On Mon, 3 Jun 2019 at 19:21, John Snow <jsnow@redhat.com> wrote:
>>> I get it, we don't want to require Python 3.8 because some dev wanted
>>> assignment conditionals -- but we're talking about Python 2 here, which
>>> suffers its EOL by the end of this calendar year.
> 
> "Not because some dev wanted assignment conditionals" is the non-reason.
> Let me spell out the reason: supporting Python 2 is expensive for us.
> As the amount of Python code grows, it gets more and more expensive.
> 
> Is this really time and effort well spent?
> 

I'd just like to clarify that you and I are arguing the same point. I
used that non-reason intentionally, arguing that we don't want a trivial
feature, we're talking about the major deprecation of an entire version
of a language.

I picked a far-fetched, very bleeding edge version (and a feature in it
that matched, sorry, been doing a lot of Python lately) to counter
Peter's point that we didn't want to support bleeding-edge stuff "just
because."

Sorry if I didn't articulate this point well.

>>> So do we think it's reasonable to drop support for Python2 for the
>>> release that comes out after Python2's EOL, or do we insist on 2x3
>>> simultaneous support for years more?
>>
>> I don't have a strong opinion on Python in particular, but
>> I think it would be nicer to avoid the "python is a special
>> snowflake" effect.
> 
> Lots of things are nice.  Limited resources dictate we can only get some
> of them.
> 
>>                    Would it really be so bad for it to just
>> be "drop it when it falls off the last LTS distro" like the
>> rest of our dependencies ?
> 
> In my experience maintaining and evolving the QAPI generators,
> supporting both Python 2 and 3 is a constant distraction that adds up
> over time.  It's all manual; we decided against adopting one of tool
> chains made for dealing with this mess.  I'd rather think about how to
> solve real user problems like deprecation information or command line
> introspection than deal with Python 2 vs. 3 arcana.
> 
> Just the other day, I flagged an innocent-looking simplification of some
> non-QAPI Python code that changed semantics subtly on Python 2, impact
> unknown.  The developer did not know.  The fact that I waste precious
> brain capacity on knowing this shit (pardon my French) is a bloody
> shame.  Well, some of this shit, because I've screwed it up myself, too.
> 
> The sooner we stop the bleeding, the better.
> 

I agree, and it looks like we have consensus that because RHEL7 has
EPEL, we can drop Python 2 support. I think.

What wasn't made clear to me yet is in which version we may do so. I
think I suggested 4.2, and nobody agreed or disagreed with that point.

--js

Re: [Qemu-devel] Deprecation policy and build dependencies

[Daniel P. Berrangé]

On Fri, May 31, 2019 at 04:24:29PM -0300, Eduardo Habkost wrote:
> Long story short: I would really like to drop support for Python
> 2 in QEMU 4.1.
> 
> What exactly prevents us from doing this?  Does our deprecation
> policy really apply to build dependencies?

In general I do *not* consider our deprecation policy to apply to
*mandatory* build dependancies. Instead our platform support policy
applies.

The rationale is that mandatory build dependancies are not something
that can be considered on a case by case basis. To build QEMU on any
given platform, *all* the mandatory build deps must be satisfied by
that platform. Increasing min version of any single mandatory, build
dep will effectively exclude a host build target.

IOW, when we drop a build target we can consider updating min version
of *all* mandatory build deps at the same time.

Where the deprecation policy could come into play is if we want to drop
an *optional* build dependancy for certain platforms. eg librbd is an
optional build dep and we might have some reason we want to increase the
min version despite it not being present in all our supported platforms.
This could be a case to mark the "rbd" feature as deprecated on certain
build platforms.


Thus to answer your python 2 question, we should ask which of our build
targets cannot support python 3 ?

Obviously we know the answer to that is RHEL-7. Except there is some
fuzziness in there because it depends on what you define "RHEL-7" to
be. There are several possible answers

 a. RHEL-7 covers only the stuff in the basic yum repos
 b. RHEL-7 covers packages in any yum repos shipped by Red Hat
 c. RHEL-7 covers packages in any yum repos shipped by Red Hat or EPEL
 d. RHEL-7 covers packages in any yum repo available for use
    with RHEL-7,  provided by any vendor

The platform support policy has not documented which of these possiblities
we're targetting.

If we consider it to mean (a), then there's no way to use py3 with RHEL-7.

With (b), (c), or (d) it is possible to get py3 available on RHEL-7 by
enabling suitable repos.

Personally I think it would be fine for use to consider (b) or (c) to be
our intended interpretation for platform support policy.

In this interpretation it is possible for developers to get Python 3 on
RHEL-7 by enabling the Red Hat Software collection repos:

  https://developers.redhat.com/products/softwarecollections/hello-world/#fndtn-windows

This implies we *can* drop python2 from QEMU *and* keep RHEL-7 as a
supported target.

Also note that the platform support policy didn't say anything about
RHEL minor updates. ie it does not distinguish RHEL-7.0 from RHEL-7.6,
despite fact that some packages get major version rebases. I think we
should clarify that we mean "latest available updates" for our supported
platforms. ie 7.6 is supported, 7.0 is *not* supported.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [Qemu-devel] Deprecation policy and build dependencies

[Daniel P. Berrangé]

On Mon, Jun 03, 2019 at 02:26:49PM +0200, Markus Armbruster wrote:
> John Snow <jsnow@redhat.com> writes:
> 
> > On 5/31/19 3:24 PM, Eduardo Habkost wrote:
> >> Long story short: I would really like to drop support for Python
> >> 2 in QEMU 4.1.
> 
> The sooner, the better, as far as I'm concerned.
> 
> >> What exactly prevents us from doing this?  Does our deprecation
> >> policy really apply to build dependencies?
> >> 
> >
> > Normally I'd say it's only nice to also follow the depreciation policy
> > for tooling as well to give people a chance to switch away, but with
> > regards to Python2, I feel like we're in the clear to drop it for the
> > first release that will happen after the Python2 doomsday clock.
> >
> > (So, probably 4.2.)
> 
> In addition to our feature deprecation policity, we have a "Supported
> build platforms" policy (commit 45b47130f4b).  The most common holdback
> is this one:
> 
>     For distributions with long-lifetime releases, the project will aim
>     to support the most recent major version at all times. Support for
>     the previous major version will be dropped 2 years after the new
>     major version is released. For the purposes of identifying supported
>     software versions, the project will look at RHEL, Debian, Ubuntu
>     LTS, and SLES distros. Other long-lifetime distros will be assumed
>     to ship similar software versions.
> 
> RHEL-7 has Python 3 only in EPEL.  RHEL-8 came out last month.  Unless
> we interpret our policy to include EPEL, this means supporting Python 2
> for some 16 months after upstream Python retires it.  My personal
> opinion: nuts.

We've not said whether this refers to only base repos, or whether addon
repos are accepted. IMHO, we are reasonably justified in saying RHEL-7
as a build platform covers any repo provided by Red Hat, which would
give us Python3 via software collections. I think it would be reasonable
to also state it covers EPEL, since EPEL is such a commonly used repo
with RHEL.

IOW, I don't think RHEL-7 support as a build platform blocks us from
dropping py2. We merely need to tweak our build platforms doc to clarify
our intent wrt add-on yum repos.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [Qemu-devel] Deprecation policy and build dependencies

[Eduardo Habkost]

On Wed, Jun 05, 2019 at 04:44:03PM +0100, Daniel P. Berrangé wrote:
[...]
> Thus to answer your python 2 question, we should ask which of our build
> targets cannot support python 3 ?
> 
> Obviously we know the answer to that is RHEL-7. Except there is some
> fuzziness in there because it depends on what you define "RHEL-7" to
> be. There are several possible answers
> 
>  a. RHEL-7 covers only the stuff in the basic yum repos
>  b. RHEL-7 covers packages in any yum repos shipped by Red Hat
>  c. RHEL-7 covers packages in any yum repos shipped by Red Hat or EPEL
>  d. RHEL-7 covers packages in any yum repo available for use
>     with RHEL-7,  provided by any vendor
> 
> The platform support policy has not documented which of these possiblities
> we're targetting.
> 
> If we consider it to mean (a), then there's no way to use py3 with RHEL-7.
> 
> With (b), (c), or (d) it is possible to get py3 available on RHEL-7 by
> enabling suitable repos.
> 
> Personally I think it would be fine for use to consider (b) or (c) to be
> our intended interpretation for platform support policy.

(c) sounds like the best option, to me.  Do we have any
reason to prefer (b) instead of (c)?

> 
> In this interpretation it is possible for developers to get Python 3 on
> RHEL-7 by enabling the Red Hat Software collection repos:
> 
>   https://developers.redhat.com/products/softwarecollections/hello-world/#fndtn-windows
> 
> This implies we *can* drop python2 from QEMU *and* keep RHEL-7 as a
> supported target.
> 
> Also note that the platform support policy didn't say anything about
> RHEL minor updates. ie it does not distinguish RHEL-7.0 from RHEL-7.6,
> despite fact that some packages get major version rebases. I think we
> should clarify that we mean "latest available updates" for our supported
> platforms. ie 7.6 is supported, 7.0 is *not* supported.

Agreed.

-- 
Eduardo

Re: [Qemu-devel] Deprecation policy and build dependencies

[Eduardo Habkost]

On Wed, Jun 05, 2019 at 04:50:06PM +0100, Daniel P. Berrangé wrote:
> On Mon, Jun 03, 2019 at 02:26:49PM +0200, Markus Armbruster wrote:
> > John Snow <jsnow@redhat.com> writes:
> > 
> > > On 5/31/19 3:24 PM, Eduardo Habkost wrote:
> > >> Long story short: I would really like to drop support for Python
> > >> 2 in QEMU 4.1.
> > 
> > The sooner, the better, as far as I'm concerned.
> > 
> > >> What exactly prevents us from doing this?  Does our deprecation
> > >> policy really apply to build dependencies?
> > >> 
> > >
> > > Normally I'd say it's only nice to also follow the depreciation policy
> > > for tooling as well to give people a chance to switch away, but with
> > > regards to Python2, I feel like we're in the clear to drop it for the
> > > first release that will happen after the Python2 doomsday clock.
> > >
> > > (So, probably 4.2.)
> > 
> > In addition to our feature deprecation policity, we have a "Supported
> > build platforms" policy (commit 45b47130f4b).  The most common holdback
> > is this one:
> > 
> >     For distributions with long-lifetime releases, the project will aim
> >     to support the most recent major version at all times. Support for
> >     the previous major version will be dropped 2 years after the new
> >     major version is released. For the purposes of identifying supported
> >     software versions, the project will look at RHEL, Debian, Ubuntu
> >     LTS, and SLES distros. Other long-lifetime distros will be assumed
> >     to ship similar software versions.
> > 
> > RHEL-7 has Python 3 only in EPEL.  RHEL-8 came out last month.  Unless
> > we interpret our policy to include EPEL, this means supporting Python 2
> > for some 16 months after upstream Python retires it.  My personal
> > opinion: nuts.
> 
> We've not said whether this refers to only base repos, or whether addon
> repos are accepted. IMHO, we are reasonably justified in saying RHEL-7
> as a build platform covers any repo provided by Red Hat, which would
> give us Python3 via software collections. I think it would be reasonable
> to also state it covers EPEL, since EPEL is such a commonly used repo
> with RHEL.
> 
> IOW, I don't think RHEL-7 support as a build platform blocks us from
> dropping py2. We merely need to tweak our build platforms doc to clarify
> our intent wrt add-on yum repos.

If we clarify the docs in QEMU 4.1, is there anything that
prevents us from removing Python 2 support in QEMU 4.1 too?

-- 
Eduardo

Re: [Qemu-devel] Deprecation policy and build dependencies

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 945 bytes --]

On 6/5/19 3:13 PM, Eduardo Habkost wrote:

>> IOW, I don't think RHEL-7 support as a build platform blocks us from
>> dropping py2. We merely need to tweak our build platforms doc to clarify
>> our intent wrt add-on yum repos.
> 
> If we clarify the docs in QEMU 4.1, is there anything that
> prevents us from removing Python 2 support in QEMU 4.1 too?

My take (but not definitive): if we have any CI setups that are testing
RHEL 7 without software collections and/or EPEL, then save Python 2
removal for 4.2 to give us time to update CI setups. But if all of our
CI setups are already fine, and we clarify the docs, then I'm all for
getting rid of Python 2 support in 4.1.

Similarly, if we are going to outlaw in-tree builds, let's get that done
in 4.1 instead of waiting yet another release.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [Qemu-devel] Deprecation policy and build dependencies

[Eduardo Habkost]

On Wed, Jun 05, 2019 at 03:42:39PM -0500, Eric Blake wrote:
> On 6/5/19 3:13 PM, Eduardo Habkost wrote:
> 
> >> IOW, I don't think RHEL-7 support as a build platform blocks us from
> >> dropping py2. We merely need to tweak our build platforms doc to clarify
> >> our intent wrt add-on yum repos.
> > 
> > If we clarify the docs in QEMU 4.1, is there anything that
> > prevents us from removing Python 2 support in QEMU 4.1 too?
> 
> My take (but not definitive): if we have any CI setups that are testing
> RHEL 7 without software collections and/or EPEL, then save Python 2
> removal for 4.2 to give us time to update CI setups. But if all of our
> CI setups are already fine, and we clarify the docs, then I'm all for
> getting rid of Python 2 support in 4.1.

If we do this soon, CI system owners will have at least 9 weeks
to fix them before 4.1.0 is released.

> 
> Similarly, if we are going to outlaw in-tree builds, let's get that done
> in 4.1 instead of waiting yet another release.

I'm missing the context on this.  Is this from a separate discussion?

-- 
Eduardo

Re: [Qemu-devel] Deprecation policy and build dependencies

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 1623 bytes --]

On 6/5/19 3:49 PM, Eduardo Habkost wrote:
> On Wed, Jun 05, 2019 at 03:42:39PM -0500, Eric Blake wrote:
>> On 6/5/19 3:13 PM, Eduardo Habkost wrote:
>>
>>>> IOW, I don't think RHEL-7 support as a build platform blocks us from
>>>> dropping py2. We merely need to tweak our build platforms doc to clarify
>>>> our intent wrt add-on yum repos.
>>>
>>> If we clarify the docs in QEMU 4.1, is there anything that
>>> prevents us from removing Python 2 support in QEMU 4.1 too?
>>
>> My take (but not definitive): if we have any CI setups that are testing
>> RHEL 7 without software collections and/or EPEL, then save Python 2
>> removal for 4.2 to give us time to update CI setups. But if all of our
>> CI setups are already fine, and we clarify the docs, then I'm all for
>> getting rid of Python 2 support in 4.1.
> 
> If we do this soon, CI system owners will have at least 9 weeks
> to fix them before 4.1.0 is released.
> 
>>
>> Similarly, if we are going to outlaw in-tree builds, let's get that done
>> in 4.1 instead of waiting yet another release.
> 
> I'm missing the context on this.  Is this from a separate discussion?

Yes, separate threads (for example, this one:
https://lists.gnu.org/archive/html/qemu-devel/2019-03/msg02553.html),
but related in that it is a build change that would affect existing
development and CI. If we're going to make life harder by pulling the
plug on old ways, we might as well pull the plug on multiple things at once.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [Qemu-devel] Deprecation policy and build dependencies

[Markus Armbruster]

Eric Blake <eblake@redhat.com> writes:

> On 6/5/19 3:13 PM, Eduardo Habkost wrote:
>
>>> IOW, I don't think RHEL-7 support as a build platform blocks us from
>>> dropping py2. We merely need to tweak our build platforms doc to clarify
>>> our intent wrt add-on yum repos.
>> 
>> If we clarify the docs in QEMU 4.1, is there anything that
>> prevents us from removing Python 2 support in QEMU 4.1 too?
>
> My take (but not definitive): if we have any CI setups that are testing
> RHEL 7 without software collections and/or EPEL, then save Python 2
> removal for 4.2 to give us time to update CI setups. But if all of our
> CI setups are already fine, and we clarify the docs, then I'm all for
> getting rid of Python 2 support in 4.1.

There's still time to update CI setups without undue haste.  But I agree
we don't want to lose CI even temporarily just to expedite getting rid
of Python 2.

> Similarly, if we are going to outlaw in-tree builds, let's get that done
> in 4.1 instead of waiting yet another release.

For that we need patches.

Re: [Qemu-devel] Deprecation policy and build dependencies

[Daniel P. Berrangé]

On Wed, Jun 05, 2019 at 03:42:39PM -0500, Eric Blake wrote:
> On 6/5/19 3:13 PM, Eduardo Habkost wrote:
> 
> >> IOW, I don't think RHEL-7 support as a build platform blocks us from
> >> dropping py2. We merely need to tweak our build platforms doc to clarify
> >> our intent wrt add-on yum repos.
> > 
> > If we clarify the docs in QEMU 4.1, is there anything that
> > prevents us from removing Python 2 support in QEMU 4.1 too?
> 
> My take (but not definitive): if we have any CI setups that are testing
> RHEL 7 without software collections and/or EPEL, then save Python 2
> removal for 4.2 to give us time to update CI setups. But if all of our
> CI setups are already fine, and we clarify the docs, then I'm all for
> getting rid of Python 2 support in 4.1.

The centos7 dockerfile will need to add the extra repo to pull
this in. I don't see any issue with that getting fixed in this
cycle.

As for any CI maintained by third parties outside QEMU tree,
they'll just have to adapt themselves. It is not difficult to
add repos, so again I don't see a big reason to delay. If we
delay chances are they won't bother to update their CI at all
until we make the py3 change in 4.2 anyway. 

> Similarly, if we are going to outlaw in-tree builds, let's get that done
> in 4.1 instead of waiting yet another release.

I don't think there was any hard objection, someone just needs to write
the patch...

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [Qemu-devel] Deprecation policy and build dependencies

[Daniel P. Berrangé]

On Wed, Jun 05, 2019 at 03:13:08PM -0300, Eduardo Habkost wrote:
> On Wed, Jun 05, 2019 at 04:44:03PM +0100, Daniel P. Berrangé wrote:
> [...]
> > Thus to answer your python 2 question, we should ask which of our build
> > targets cannot support python 3 ?
> > 
> > Obviously we know the answer to that is RHEL-7. Except there is some
> > fuzziness in there because it depends on what you define "RHEL-7" to
> > be. There are several possible answers
> > 
> >  a. RHEL-7 covers only the stuff in the basic yum repos
> >  b. RHEL-7 covers packages in any yum repos shipped by Red Hat
> >  c. RHEL-7 covers packages in any yum repos shipped by Red Hat or EPEL
> >  d. RHEL-7 covers packages in any yum repo available for use
> >     with RHEL-7,  provided by any vendor
> > 
> > The platform support policy has not documented which of these possiblities
> > we're targetting.
> > 
> > If we consider it to mean (a), then there's no way to use py3 with RHEL-7.
> > 
> > With (b), (c), or (d) it is possible to get py3 available on RHEL-7 by
> > enabling suitable repos.
> > 
> > Personally I think it would be fine for use to consider (b) or (c) to be
> > our intended interpretation for platform support policy.
> 
> (c) sounds like the best option, to me.  Do we have any
> reason to prefer (b) instead of (c)?

Depends how flexible we want to be. Some enterprise organizations will
not allow use of 3rd party repos, even EPEL, only permitting to consume
software provided by Red Hat official repos.  I'm not too bothered though
as I doubt it will be a major problem with likely QEMU contributors. Those
restrictive organizations are not likely to allow developers to be involved
in upstream communities in the first place, nor consume releases direct
from upstream.

IOW, I think (c) is fine to allow maximum flexibility for upstream.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|