Survey Results - Part 3 : Improving Xen Project Governance and Conventions

Survey Results - Part 3 : Improving Xen Project Governance and Conventions

[Lars Kurth]

Hi all, 

please find results of part 2, following on from 
- Part 1 : http://lists.xenproject.org/archives/html/xen-devel/2015-10/msg00354.html 
- Part 2 : http://lists.xenproject.org/archives/html/xen-devel/2015-10/msg01871.html

Part 4 will be omitted : too many different opinions.

Results and comments are marked with | under each question. Followed by an  analysis, which is marked with !

Best Regards
Lars


== 3) Other related Governance Issues  --


There are a number of other shortfalls in http://www.xenproject.org/governance.html 
that have been raised in the past. Namely ...

 * <u>Voting for Governance Changes</u>

   http://lists.xen.org/archives/html/xen-devel/2014-11/msg01548.html raised 
   the issue that voting for governance changes is tied to committer status. 
   One issue with the current set-up is that the XAPI sub-project for example 
   does not differentiate between maintainers and committers and Mirage OS is 
   similar (aka all maintainers are also committers). Thus, the number of 
   qualifying XAPI votes (and in future Mirage OS votes) is significantly 
   higher than in the Hypervisor. This could create issues when voting for 
   global policy changes once Mirage OS graduates.

Q3.1: How would you feel about changing the pool of people that are allowed to 
vote on governance changes. Possible groups that may be allowed to vote could 
be 

a) Committers (now), 
b) Committers and Maintainers (possibly after changing criteria for both), 
c) A new, yet to be defined group, which constitutes the project leadership 
  (see Q2.6), 
d) An approach which makes key contributors eligible based on the previous 
  year’s contribution (e.g. the top X contributors, each contributor who 
  contributed more than X% last year, …) - we could limit the number of 
  eligible voters based on sub-project size for global policy votes. 
e) Some combination of c) and d) 

| We had people voting for multiple options equally. So this result has
| to be considered with some care.
|
| a)           3   
| b)           3 
| c)           5
| d)           2 
| e)           1
| No opinion:  2

| Sub-projects should have a single vote; if they have multiple committers, 
| they should vote amongst themselves to determine how to vote on a project 
| wide change. 

! I think the proposal for Xen Project wide issues also makes sense: aka 
! "Sub-projects should have a single vote" which are then tallied up and
! follow all the other rules.

| Contribution-weighted votes would be nice, but you would need to find a 
| way to not only count submissions or commits, but also code reviews.

| I'd prefer a) with maybe the possibility to let the committers decide
| whether to ask a wider audience regarding a specific change.

| In terms of our existing people, this should be all the committers
| plus a small number of the existing maintainers.

| I would be in favor of separating committership from project leadership;
| but I haven't seen a proposal which I actually like better than a.

| I think a publicly elected committee of X people would be the best
| option. I have however not clear ideas on who should be allowed to vote.
| I guess doing it by contributions is a step forward, but we should be
| very careful with this, so that people don't start sending trivial
| patches in order to get a vote.
| 
| Should we also then consider other projects which are related to Xen but
| it's code doesn't reside inside of the Xen repository (Qemu, Linux...)?

! = Summary =
!
! We seem to have a clear majority to expand the pool of voters, but no
! real consensus on whom to add. There is a slight preference towards c)
! aka a group representing the project leadership consisting of committers
! plus some additional people. I think I would like to get some feedback
! on this thread and then make a concrete proposal. Elections are not unusual
! in open source projects when it comes to boards, but not so common for 
! the leadership: there is also a chicken and egg problem (in other words,
! who elects the leadership).

Please state a preference and explain why. Note that a) and b) don’t fit XAPI, 
Mirage OS, ... This means that we may not be able to retain a single 
governance document across all projects. A fudge could be for each project to 
nominate/elect a leadership using project local criteria (aka c), which then 
vote on behalf of that subproject. Also consider, what would encourage 
community members to step up getting more actively engaged with reviews and 
other areas where we have bottlenecks today.

 * <u>Blocking Votes</u>

   We had several cases, where a voter wanted to express disagreement with a 
   proposal, but did not want to block a vote. I wanted to suggest an 
   approach that we used very successfully in Event Program Management 
   Committees. 

   It is based on splitting -1 into
    -1 : disagree, but don’t care enough to argue against it
    -2 : disagree, but feel strongly enough to argue against it
   and +1 into 
    +1: agree, but don’t care enough to argue for it
    +2: agree, but care strongly enough to argue for it

   Note that the -2, ... +2 is merely a shorthand and is not intended for 
   counting and could be replaced by some other shorthand.

Q3.2: Do you think such a proposal makes sense and is workable? 

| 11 / 12 Agree
| 0 / 12 Disagree
| 1 / 12 Don't know or have no opinion

| Yes, though I think the current system of voting 0 and explaining a
| weak preference in words does just as well.
|
| Note that there were a few comments along those lines.

| Yes, this ought to be workable. 

| I question whether the current model of a single negative vote blocking 
| a vote is a good one: The UN Security Council is good example of
| why one shouldn't allow single entities to veto everything.

! This seems to be clear cut 

 * <u>Majority</u>

 Every -1 vote today, is essentially a veto. With a growing pool of votes, it 
 becomes possible for an individual to block progress. Without a referee (see 
 Q2.6), who we are currently assuming in our governance will resolve a 
 conflict, we could find ourselves in a limbo.

Q3.3: Do you think introducing a deadlock resolution approach would make 
sense, if designed to be used in exceptional circumstances only. Would a 
majority based approach make sense?

| 10 / 12 Agree
| 0 / 12 Disagree
| 2 / 12 Don't know or have no opinion

| If we extend the voting to maintainers or expand the number of
| committers, then yes, but we should not use a simple majority -- after
| all we want to reach consensus.  We could require, e.g., a majority in
| favour and no more than 20% (strongly) opposed.

| If we allow for deadlocks to occur, we need such an approach

| I'm a little bit wary of designing new forms of government and voting etc,
| since the law of unintended consequences often comes into play.

| I think that we should abolish the blocking veto in governance votes.
|
| As a first step it should be possible for a voter to vote against
| something without vetoing it.

| I thought the goal was always for the committers to go along with the
| consensus of the community, and then to fall back to a majority vote if
| consensus failed.

! On the point above, I think this was the case for some things (e.g.
! process) but not for others, such as appointing new committers. In
! practice this is sort of broken, I think. Even if this was the goal,
! what would happen if a committer persistently votes against the 
! consensus.

! = Summary =
!
! There seems to be a clear consensus against the current veto model. From
! the comments there was no clear case for a simple majority, but for a higher
! bar of consensus (2/3, 75% or <20% objections) depending on the size
! of the group.
!
! So I suppose again, I will let this run for a bit and then get some 
! feedback to be incorporated into a proposal.


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Changing the voting model, Was: Survey Results - Part 3 : Improving Xen Project Governance and Conventions

[Stefano Stabellini]

[-- Attachment #1: Type: text/plain, Size: 1835 bytes --]

On Fri, 16 Oct 2015, Lars Kurth wrote:
>  * <u>Blocking Votes</u>
>
>    We had several cases, where a voter wanted to express disagreement with a
>    proposal, but did not want to block a vote. I wanted to suggest an
>    approach that we used very successfully in Event Program Management
>    Committees.
>
>    It is based on splitting -1 into
>     -1 : disagree, but don’t care enough to argue against it
>     -2 : disagree, but feel strongly enough to argue against it
>    and +1 into
>     +1: agree, but don’t care enough to argue for it
>     +2: agree, but care strongly enough to argue for it
>
>    Note that the -2, ... +2 is merely a shorthand and is not intended for
>    counting and could be replaced by some other shorthand.
>
> Q3.2: Do you think such a proposal makes sense and is workable?
>
> | 11 / 12 Agree
> | 0 / 12 Disagree
> | 1 / 12 Don't know or have no opinion

[...]

> ! = Summary =
> !
> ! There seems to be a clear consensus against the current veto model. From
> ! the comments there was no clear case for a simple majority, but for a higher
> ! bar of consensus (2/3, 75% or <20% objections) depending on the size
> ! of the group.
> !
> ! So I suppose again, I will let this run for a bit and then get some
> ! feedback to be incorporated into a proposal.

This seems to be a good place to start to make changes based on the
results from the survey.

Everybody seems to agree on changing the voting model from consensus to
some kind of majority vote among committers. I agree with Lars: we
should gather a few proposals then call a vote to see which one is the
best for us.

I think we should avoid counting abstentions and blank votes, only
positive and negative votes by committers count, and go with a simple
majority.

What do you think?

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Changing the voting model, Was: Survey Results - Part 3 : Improving Xen Project Governance and Conventions

[Lars Kurth]

> On 21 Oct 2015, at 17:02, Stefano Stabellini <stefano.stabellini@eu.citrix.com> wrote:
> 
> On Fri, 16 Oct 2015, Lars Kurth wrote:
>> * <u>Blocking Votes</u>
>> 
>>   We had several cases, where a voter wanted to express disagreement with a 
>>   proposal, but did not want to block a vote. I wanted to suggest an 
>>   approach that we used very successfully in Event Program Management 
>>   Committees. 
>> 
>>   It is based on splitting -1 into
>>    -1 : disagree, but don’t care enough to argue against it
>>    -2 : disagree, but feel strongly enough to argue against it
>>   and +1 into 
>>    +1: agree, but don’t care enough to argue for it
>>    +2: agree, but care strongly enough to argue for it
>> 
>>   Note that the -2, ... +2 is merely a shorthand and is not intended for 
>>   counting and could be replaced by some other shorthand.
>> 
>> Q3.2: Do you think such a proposal makes sense and is workable? 
>> 
>> | 11 / 12 Agree
>> | 0 / 12 Disagree
>> | 1 / 12 Don't know or have no opinion
> 
> [...]
> 
>> ! = Summary =
>> !
>> ! There seems to be a clear consensus against the current veto model. From
>> ! the comments there was no clear case for a simple majority, but for a higher
>> ! bar of consensus (2/3, 75% or <20% objections) depending on the size
>> ! of the group.
>> !
>> ! So I suppose again, I will let this run for a bit and then get some 
>> ! feedback to be incorporated into a proposal.
> 
> This seems to be a good place to start to make changes based on the
> results from the survey.
> 
> Everybody seems to agree on changing the voting model from consensus to
> some kind of majority vote among committers. I agree with Lars: we
> should gather a few proposals then call a vote to see which one is the
> best for us.
> 
> I think we should avoid counting abstentions and blank votes, only
> positive and negative votes by committers count, and go with a simple
> majority.
> 
> What do you think?

To be honest, ideally I would prefer to also look at who can vote as part of this topic, rather than making two separate changes within a few months, if that can be avoided. In particular, a higher threshold for a small group (5 committers) in effect still amounts to a veto. 

<20% objections of 6 committers, is 1 vote against = retains a veto in practice
<25% objections of 6 committers, is 1.25 votes against would count as a veto depending on rounding 
<33% of objections of 6 committers, is 1.66 votes against

A majority vote would require 4 positive votes if all committers voted. And 3 positive votes if 4 or 5 committers voted.

In practice, for most issues we typically had 4 votes. Which sort of retains the veto,
If fewer committers vote, the arithmetics look even worse.

Looking at 

> Q3.1: How would you feel about changing the pool of people that are allowed to 
> vote on governance changes. Possible groups that may be allowed to vote could 
> be 
> 
> a) Committers (now), 
> b) Committers and Maintainers (possibly after changing criteria for both), 
> c) A new, yet to be defined group, which constitutes the project leadership 
>  (see Q2.6), 
> d) An approach which makes key contributors eligible based on the previous 
>  year’s contribution (e.g. the top X contributors, each contributor who 
>  contributed more than X% last year, …) - we could limit the number of 
>  eligible voters based on sub-project size for global policy votes. 
> e) Some combination of c) and d) 
> 
> | We had people voting for multiple options equally. So this result has
> | to be considered with some care.
> |
> | a)           3   
> | b)           3 
> | c)           5
> | d)           2 
> | e)           1
> | No opinion:  2

and doing the sums. e.g. ...

a + No opinion = 5 ... let's count these as against increasing the pool of voters
b + c + d + e = 13 ... let's count these in favour of increasing the pool of voters

... I would argue that we have a clear majority to increase the pool of voters on policy and governance changes. 

The question is 

A) How big should the pool of people qualifying to vote on governance changes be? For simplicity, let's just call that set of people "leadership team". I think it is clear that active Committers should be part of the team. I would say that the current Release Manager should be included also. Around 10-12, 
seems a reasonable number.

B) From what I can tell, there are no objections to include a subset of maintainers into the "leadership team". And it would make sense. 

C) There seems to be a bias against automatically adding maintainers based on some rules (e.g. contribution stats) on the grounds that such a system can be gamed

D) Given C, the only option is then to go for some nomination / election based approach, where "X maintainers" are elected to be part of the "leadership team" for a term of "Y years" (I would recommend that Y is > 1 year; maybe 2). Maybe with a provision that if someone stops contributing, for a set period of time, they drop out of the "leadership team".

One concern with voting is that a vendor could take over the project: maybe there should be limits to how many people of a single vendor can be in the leadership team.

All that is missing is a concrete proposal.  

Best Regards
Lars





_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Changing the voting model, Was: Survey Results - Part 3 : Improving Xen Project Governance and Conventions

[George Dunlap]

On Wed, Oct 21, 2015 at 4:02 PM, Stefano Stabellini
<stefano.stabellini@eu.citrix.com> wrote:
>> ! = Summary =
>> !
>> ! There seems to be a clear consensus against the current veto model. From
>> ! the comments there was no clear case for a simple majority, but for a higher
>> ! bar of consensus (2/3, 75% or <20% objections) depending on the size
>> ! of the group.
>> !
>> ! So I suppose again, I will let this run for a bit and then get some
>> ! feedback to be incorporated into a proposal.
>
> This seems to be a good place to start to make changes based on the
> results from the survey.
>
> Everybody seems to agree on changing the voting model from consensus to
> some kind of majority vote among committers. I agree with Lars: we
> should gather a few proposals then call a vote to see which one is the
> best for us.
>
> I think we should avoid counting abstentions and blank votes, only
> positive and negative votes by committers count, and go with a simple
> majority.
>
> What do you think?

I'm a bit confused by this discussion.  Are we talking about checking
in code, or about project-wide policy decisions?

For patch review, in general, no patch goes in while there are any
outstanding objections, regardless of who is doing the objecting
(consensus).  But it is permissible for maintainers of the code being
changed to override the person objecting, if they are not a
maintainer.  It is understood that this should be done only after a
reasonable discussion, and only if the maintainer thinks there is a
good reason to do so.

According to the governance doc [1], we are supposed to *try* to form
a consensus; but in the case that a consensus cannot be formed, things
fall back to a majority vote among the committers.  This parallels the
above example with patch review, but at a community-wide level: we
should first try to have a reasonable discussion, but if we cannot
achieve consensus, then a majority of committers can override the
objector (even if she is a committer).

There's some verbiage in there about this being a "last resort" in the
case that the community "becomes paralyzed", but I think we shouldn't
necessarily see it as quite so extreme: it should simply be understood
that such an override should only happen after a reasonable
discussion, and the override should only be given if the committers
think there is a good reason to do so.

The only "veto model" we have is where there is overlapping
maintainership: if a single maintainer says "no", then in theory that
is a veto for a particular patch to go in.  I think having a major
argument between maintainers would be quite exceptional, and if we hit
this situation then we clearly have a breakdown of relationships that
needs to be addressed with the individuals (in the last resort by
asking someone to step down as a maintainer), not by clarifying the
governance model.

 -George

[1] http://www.xenproject.org/governance.html

Re: Changing the voting model, Was: Survey Results - Part 3 : Improving Xen Project Governance and Conventions

[Stefano Stabellini]

On Thu, 22 Oct 2015, George Dunlap wrote:
> On Wed, Oct 21, 2015 at 4:02 PM, Stefano Stabellini
> <stefano.stabellini@eu.citrix.com> wrote:
> >> ! = Summary =
> >> !
> >> ! There seems to be a clear consensus against the current veto model. From
> >> ! the comments there was no clear case for a simple majority, but for a higher
> >> ! bar of consensus (2/3, 75% or <20% objections) depending on the size
> >> ! of the group.
> >> !
> >> ! So I suppose again, I will let this run for a bit and then get some
> >> ! feedback to be incorporated into a proposal.
> >
> > This seems to be a good place to start to make changes based on the
> > results from the survey.
> >
> > Everybody seems to agree on changing the voting model from consensus to
> > some kind of majority vote among committers. I agree with Lars: we
> > should gather a few proposals then call a vote to see which one is the
> > best for us.
> >
> > I think we should avoid counting abstentions and blank votes, only
> > positive and negative votes by committers count, and go with a simple
> > majority.
> >
> > What do you think?
> 
> I'm a bit confused by this discussion.  Are we talking about checking
> in code, or about project-wide policy decisions?
> 
> For patch review, in general, no patch goes in while there are any
> outstanding objections, regardless of who is doing the objecting
> (consensus).  But it is permissible for maintainers of the code being
> changed to override the person objecting, if they are not a
> maintainer.  It is understood that this should be done only after a
> reasonable discussion, and only if the maintainer thinks there is a
> good reason to do so.
> 
> According to the governance doc [1], we are supposed to *try* to form
> a consensus; but in the case that a consensus cannot be formed, things
> fall back to a majority vote among the committers.  This parallels the
> above example with patch review, but at a community-wide level: we
> should first try to have a reasonable discussion, but if we cannot
> achieve consensus, then a majority of committers can override the
> objector (even if she is a committer).
> 
> There's some verbiage in there about this being a "last resort" in the
> case that the community "becomes paralyzed", but I think we shouldn't
> necessarily see it as quite so extreme: it should simply be understood
> that such an override should only happen after a reasonable
> discussion, and the override should only be given if the committers
> think there is a good reason to do so.
> 
> The only "veto model" we have is where there is overlapping
> maintainership: if a single maintainer says "no", then in theory that
> is a veto for a particular patch to go in.  I think having a major
> argument between maintainers would be quite exceptional, and if we hit
> this situation then we clearly have a breakdown of relationships that
> needs to be addressed with the individuals (in the last resort by
> asking someone to step down as a maintainer), not by clarifying the
> governance model.

What is written on the governance is not what I was expecting from the
quoted text above and my experience on xen-devel.

Actually I reread the governance before writing my email yesterday, but my
conclusion was that nobody likes to resort to the "last resort",
therefore things get stalled during the initial consensus phase. I don't
recall seeing the "last resort" vote ever being called.

Do you think that this is what is happening? Maybe we just need to
follow the current governance model a bit more closely and invoke the
"last resort" vote more often?

Re: Changing the voting model, Was: Survey Results - Part 3 : Improving Xen Project Governance and Conventions

[Jan Beulich]

>>> On 22.10.15 at 18:20, <stefano.stabellini@eu.citrix.com> wrote:
> What is written on the governance is not what I was expecting from the
> quoted text above and my experience on xen-devel.
> 
> Actually I reread the governance before writing my email yesterday, but my
> conclusion was that nobody likes to resort to the "last resort",
> therefore things get stalled during the initial consensus phase. I don't
> recall seeing the "last resort" vote ever being called.
> 
> Do you think that this is what is happening? Maybe we just need to
> follow the current governance model a bit more closely and invoke the
> "last resort" vote more often?

Mind giving a couple of examples of such stalls, where you would
expect the "last resort" to be needed?

Jan

Re: Changing the voting model, Was: Survey Results - Part 3 : Improving Xen Project Governance and Conventions

[Lars Kurth]

> On 22 Oct 2015, at 16:04, George Dunlap <dunlapg@umich.edu> wrote:
> 
> On Wed, Oct 21, 2015 at 4:02 PM, Stefano Stabellini
> <stefano.stabellini@eu.citrix.com> wrote:
>>> ! = Summary =
>>> !
>>> ! There seems to be a clear consensus against the current veto model. From
>>> ! the comments there was no clear case for a simple majority, but for a higher
>>> ! bar of consensus (2/3, 75% or <20% objections) depending on the size
>>> ! of the group.
>>> !
>>> ! So I suppose again, I will let this run for a bit and then get some
>>> ! feedback to be incorporated into a proposal.
>> 
>> This seems to be a good place to start to make changes based on the
>> results from the survey.
>> 
>> Everybody seems to agree on changing the voting model from consensus to
>> some kind of majority vote among committers. I agree with Lars: we
>> should gather a few proposals then call a vote to see which one is the
>> best for us.
>> 
>> I think we should avoid counting abstentions and blank votes, only
>> positive and negative votes by committers count, and go with a simple
>> majority.
>> 
>> What do you think?
> 
> I'm a bit confused by this discussion.  Are we talking about checking
> in code, or about project-wide policy decisions?

I was mainly looking at this from the viewpoint of project-wide (or-subproject wide) policy decision and giving more people which are active in the project a stake in policy decisions. I have noticed that very few people outside committers today engage in these discussions. What I don't know is whether this is because the formal decision in the end lies with the committers and whether more people would engage, if they had a formal say.

> For patch review, in general, no patch goes in while there are any
> outstanding objections, regardless of who is doing the objecting
> (consensus).  But it is permissible for maintainers of the code being
> changed to override the person objecting, if they are not a
> maintainer.  It is understood that this should be done only after a
> reasonable discussion, and only if the maintainer thinks there is a
> good reason to do so.
> 
> According to the governance doc [1], we are supposed to *try* to form
> a consensus; but in the case that a consensus cannot be formed, things
> fall back to a majority vote among the committers.  This parallels the
> above example with patch review, but at a community-wide level: we
> should first try to have a reasonable discussion, but if we cannot
> achieve consensus, then a majority of committers can override the
> objector (even if she is a committer).

I am assuming you refer to "Thus, as a last resort when consensus cannot be achieved on a question internal to a project, the final decision will be made by a private majority vote amongst the committers and project lead. If the vote is tied, the project lead gets an extra vote to break the tie."

If we had an active project lead, this would be good enough. In reality we do not have an active project lead any more. So we could theoretically end up in with a tie. There is also a provision for the Advisory Board to provide the casting vote (although not on project internal matters). I really wouldn't want to end up in a situation, where the "Advisory Board" makes a casting vote. In any case, the text in "Last Resort" does not seem to be reflecting the current reality of not having an active project lead. 

In particular as in the survey we established that we prefer the status-quo of a committee based project leadership, rather than a "benevolent dictator" model, with which we started with and which the governance still assumes. Part of the reason why I kicked off the survey, was to see whether a) we had any glaring issues somewhere, b) see how we future proof the governance based on the now validated assumption that we would probably not elect a new project lead. 

I also think that "Elections and Formal Votes" in [1] is not very clear: for example, it is not really clear whether a majority say in a "Committer Election" or a "Process/Convention change" would suffice. Aka, whether the "Last Resort" section applies to formal governance related votes. In practice, we always had full consensus and often treated formal votes as if negative votes were a veto. I remember at least one cases, where Jan registered an objection, but phrased it such that he would go with a majority view. 
 
> There's some verbiage in there about this being a "last resort" in the
> case that the community "becomes paralyzed", but I think we shouldn't
> necessarily see it as quite so extreme: it should simply be understood
> that such an override should only happen after a reasonable
> discussion, and the override should only be given if the committers
> think there is a good reason to do so.

And it appears to me that we have not had any incidents and issues in this area.

> The only "veto model" we have is where there is overlapping
> maintainership: if a single maintainer says "no", then in theory that
> is a veto for a particular patch to go in.  I think having a major
> argument between maintainers would be quite exceptional, and if we hit
> this situation then we clearly have a breakdown of relationships that
> needs to be addressed with the individuals (in the last resort by
> asking someone to step down as a maintainer), not by clarifying the
> governance model.

I agree. I still claim though that sections "roles / project lead", "conflict resolution" and "elections and formal votes" would probably benefit from being cleaned up and made clear. And maybe, we can take this as an opportunity to come up with an approach that better encourages contributions (in the general sense, not just code contributions). I also remember that Konrad pointed us to the "Code of Conflict" that Linux has, but that does not seem sufficient in light of Sarah Sharp's recent resignation as a Linux Maintainer.

Best Regards
Lars