[RFC] Xen PV Drivers Lifecycle

[RFC] Xen PV Drivers Lifecycle

[Stefano Stabellini]

Hi all,

as you know, we have an issue with the speed of review and acceptance of
new PV drivers. In a discussion among committers, George wrote an email
with a short proposal to clarify the development lifecycle of new PV
drivers and the different expectations at each stage of the process. I
took that email, polished it and turned it into markdown. Here it is.

---

# Xen PV Drivers lifecycle

## Purpose

Getting new PV drivers accepted in Xen, upstream code bases, and ABI
stable in the quickest and most efficient way possible.


## Design Phase

The first step toward acceptance of a new PV protocol is to write a
design document and send it to xen-devel. It should cover the xenstore
handshake mechanism, the ABI, how the protocol works and anything else
which is required to write an implementation of it. The usage of C-like
structs to describe language and platform agnostic protocols is
discouraged.

An attempt should be made for the protocol ABI to be backward compatible
and OS agnostic, but, realistically, backward and cross-platform
compatibility are not fully expected at this stage.

After the high level design of the protocol has been discussed and
agreed, the document is committed to xen.git.


## Prototype Stage

The contributor sends patches to implement the PV drivers for the new
protocol to the relevant open source mailing lists, such as LKML,
qemu-devel and xen-devel. 

The code is expected to work, be good quality and faithfully implement
the spec. However, there are no promises about ABI and cross-platform
compatibility yet.

After careful review by the relevant maintainers, the code is committed
to the upstream code bases. The drivers are considered experimental.


## Production Stage

The quality of the drivers and the spec is improved. Bugs are fixed.
The protocol version is likely bumped. More testing leads to confidence
that the spec and the drivers are ready for production usage. Promises
about backward compatibility and cross-platform compatibility are
clearly spelled out.


## How to move forward from a stage to the next

The PV protocols Czar is responsible for determining the transitions
between stages. Our governance principles specify "lazy consensus" for
most things. It applies to this case too. New PV protocols should move
from one stage to the next within a reasonable time frame unless someone
has specific technical objections and voices them in a responsive
manner. 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [RFC] Xen PV Drivers Lifecycle

[Jan Beulich]

>>> On 20.12.16 at 01:47, <sstabellini@kernel.org> wrote:
> ## Design Phase
> 
> The first step toward acceptance of a new PV protocol is to write a
> design document and send it to xen-devel. It should cover the xenstore
> handshake mechanism, the ABI, how the protocol works and anything else
> which is required to write an implementation of it. The usage of C-like
> structs to describe language and platform agnostic protocols is
> discouraged.
> 
> An attempt should be made for the protocol ABI to be backward compatible
> and OS agnostic, but, realistically, backward and cross-platform
> compatibility are not fully expected at this stage.

How does backward compatibility matter for a new protocol? Is
this perhaps rather about forward compatibility provisions
(like requiring reserved fields to be zero to allow future use)?

Jan


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [RFC] Xen PV Drivers Lifecycle

[Stefano Stabellini]

On Tue, 20 Dec 2016, Jan Beulich wrote:
> >>> On 20.12.16 at 01:47, <sstabellini@kernel.org> wrote:
> > ## Design Phase
> > 
> > The first step toward acceptance of a new PV protocol is to write a
> > design document and send it to xen-devel. It should cover the xenstore
> > handshake mechanism, the ABI, how the protocol works and anything else
> > which is required to write an implementation of it. The usage of C-like
> > structs to describe language and platform agnostic protocols is
> > discouraged.
> > 
> > An attempt should be made for the protocol ABI to be backward compatible
> > and OS agnostic, but, realistically, backward and cross-platform
> > compatibility are not fully expected at this stage.
> 
> How does backward compatibility matter for a new protocol? Is
> this perhaps rather about forward compatibility provisions
> (like requiring reserved fields to be zero to allow future use)?

By "backward compatibility" I mean promising that, in 5 years time, a
new frontend will still be able to connect to a backend written in 2016.

This level of support requires an understanding of the protocol and its
subtleties which usually only comes from experience. Hence, I am
suggesting to make this kind of promises only after the code has lived
in-tree for a while and has been subject to wider testing.

Maybe I should call it cross-versions compatibility?

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [RFC] Xen PV Drivers Lifecycle

[George Dunlap]

On Tue, Dec 20, 2016 at 11:09 PM, Stefano Stabellini
<sstabellini@kernel.org> wrote:
> On Tue, 20 Dec 2016, Jan Beulich wrote:
>> >>> On 20.12.16 at 01:47, <sstabellini@kernel.org> wrote:
>> > ## Design Phase
>> >
>> > The first step toward acceptance of a new PV protocol is to write a
>> > design document and send it to xen-devel. It should cover the xenstore
>> > handshake mechanism, the ABI, how the protocol works and anything else
>> > which is required to write an implementation of it. The usage of C-like
>> > structs to describe language and platform agnostic protocols is
>> > discouraged.
>> >
>> > An attempt should be made for the protocol ABI to be backward compatible
>> > and OS agnostic, but, realistically, backward and cross-platform
>> > compatibility are not fully expected at this stage.
>>
>> How does backward compatibility matter for a new protocol? Is
>> this perhaps rather about forward compatibility provisions
>> (like requiring reserved fields to be zero to allow future use)?
>
> By "backward compatibility" I mean promising that, in 5 years time, a
> new frontend will still be able to connect to a backend written in 2016.
>
> This level of support requires an understanding of the protocol and its
> subtleties which usually only comes from experience. Hence, I am
> suggesting to make this kind of promises only after the code has lived
> in-tree for a while and has been subject to wider testing.
>
> Maybe I should call it cross-versions compatibility?

Right, so I think you mean that you design the (new) protocol such
that *future* versions can be backwards-compatible.

What if you rephrase it something like the following?

"An attempt should be made to design the ABI such that it will be OS
agnostic, that future versions will not need to introduce
backward-incompatible changes, and so on; but these are not yet hard
requirements."  (Or perhaps, "not yet first-order goals.")

 -George

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [RFC] Xen PV Drivers Lifecycle

[Stefano Stabellini]

On Wed, 28 Dec 2016, George Dunlap wrote:
> On Tue, Dec 20, 2016 at 11:09 PM, Stefano Stabellini
> <sstabellini@kernel.org> wrote:
> > On Tue, 20 Dec 2016, Jan Beulich wrote:
> >> >>> On 20.12.16 at 01:47, <sstabellini@kernel.org> wrote:
> >> > ## Design Phase
> >> >
> >> > The first step toward acceptance of a new PV protocol is to write a
> >> > design document and send it to xen-devel. It should cover the xenstore
> >> > handshake mechanism, the ABI, how the protocol works and anything else
> >> > which is required to write an implementation of it. The usage of C-like
> >> > structs to describe language and platform agnostic protocols is
> >> > discouraged.
> >> >
> >> > An attempt should be made for the protocol ABI to be backward compatible
> >> > and OS agnostic, but, realistically, backward and cross-platform
> >> > compatibility are not fully expected at this stage.
> >>
> >> How does backward compatibility matter for a new protocol? Is
> >> this perhaps rather about forward compatibility provisions
> >> (like requiring reserved fields to be zero to allow future use)?
> >
> > By "backward compatibility" I mean promising that, in 5 years time, a
> > new frontend will still be able to connect to a backend written in 2016.
> >
> > This level of support requires an understanding of the protocol and its
> > subtleties which usually only comes from experience. Hence, I am
> > suggesting to make this kind of promises only after the code has lived
> > in-tree for a while and has been subject to wider testing.
> >
> > Maybe I should call it cross-versions compatibility?
> 
> Right, so I think you mean that you design the (new) protocol such
> that *future* versions can be backwards-compatible.
> 
> What if you rephrase it something like the following?
> 
> "An attempt should be made to design the ABI such that it will be OS
> agnostic, that future versions will not need to introduce
> backward-incompatible changes, and so on; but these are not yet hard
> requirements."  (Or perhaps, "not yet first-order goals.")

sounds good to me

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [RFC] Xen PV Drivers Lifecycle

[Wei Liu]

+1 for this in principle.

I have no further comment to make. :-)

Wei.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel