[PATCH] Documentation/process: Add Linux Kernel Contribution Maturity Model

[PATCH] Documentation/process: Add Linux Kernel Contribution Maturity Model

[Theodore Ts'o]

As a follow-up to a discussion at the 2021 Maintainer's Summit on the
topic of maintainer recruitment and retention, the TAB took on the
task of creating a document which to help companies and other
organizations to grow in their ability to engage with the Linux Kernel
development community, using the Maturity Model[2] framework.

The goal is to encourage, in a management-friendly way, companies to
allow their engineers to contribute with the upstream Linux Kernel
development community, so we can grow the "talent pipeline" for
contributors to become respected leaders, and eventually kernel
maintainers.

[1] https://lwn.net/Articles/870581/
[2] https://en.wikipedia.org/wiki/Maturity_model

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Co-developed-by: Kees Cook <keescook@chromium.org>
Signed-off-by: Kees Cook <keescook@chromium.org>
Co-developed-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Acked-by: Jakub Kicinski <kuba@kernel.org>
Acked-by: Christian Brauner (Microsoft) <brauner@kernel.org>
Acked-by: Dave Hansen <dave.hansen@linux.intel.com>
---
 .../process/contribution-maturity-model.rst   | 111 ++++++++++++++++++
 Documentation/process/index.rst               |   1 +
 2 files changed, 112 insertions(+)
 create mode 100644 Documentation/process/contribution-maturity-model.rst

diff --git a/Documentation/process/contribution-maturity-model.rst b/Documentation/process/contribution-maturity-model.rst
new file mode 100644
index 000000000000..39a2b6b116ca
--- /dev/null
+++ b/Documentation/process/contribution-maturity-model.rst
@@ -0,0 +1,111 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+.. _contribution_maturity_model:
+
+========================================
+Linux Kernel Contribution Maturity Model
+========================================
+
+
+Background
+==========
+
+As a part of the 2021 Linux Kernel Maintainers’ Summit, there was a
+`discussion <https://lwn.net/Articles/870581/>`_ about the challenges in
+recruiting kernel maintainers as well as maintainer succession.  Some of
+the conclusions from that discussion included that companies which are a
+part of the Linux Kernel community need to allow engineers to be
+maintainers as part of their job, so they can grow into becoming
+respected leaders and eventually, kernel maintainers.  To support a
+strong talent pipeline, developers need to allowed and encouraged to
+take on upstream contributions such as reviewing other people’s patches,
+refactoring kernel infrastructure, and writing documentation.
+
+To that end, the Linux Foundation Technical Advisory Board (TAB)
+proposes this Linux Kernel Contribution Maturity Model. These common
+expectations for upstream community engagement aim to increase the
+influence of individual developers, increase the collaboration of
+organizations, and improve the overall health of the Linux Kernel
+ecosystem.
+
+The TAB urges organizations to continuously evaluate their Open Source
+maturity model and commit to improvements to align with this model.  To
+be effective, this evaluation should incorporate feedback from across
+the organization, including management and developers at all seniority
+levels.  In the spirit of Open Source, we encourage organizations to
+publish their evaluations and plans to improve their engagement with the
+upstream community.
+
+Level 0
+=======
+
+* Software Engineers are not allowed to contribute patches to the Linux
+  kernel.
+
+
+Level 1
+=======
+
+* Software Engineers are allowed to contribute patches to the Linux
+  kernel, either as part of their job responsibilities or on their own
+  time.
+
+Level 2
+=======
+
+* Software Engineers are expected to contribute to the Linux Kernel as
+  part of their job responsibilities.
+* Software Engineers will be supported to attend Linux-related
+  conferences as a part of their job.
+* A Software Engineer’s upstream code contributions shall be considered
+  in promotion and performance reviews.
+
+Level 3
+=======
+
+* Software Engineers are expected to review patches (including patches
+  authored by engineers from other companies) as part of their job
+  responsibilities
+* Contributing presentations or papers to Linux-related conferences are
+  considered part of an engineer’s work.
+* A Software Engineer’s community contributions shall be considered in
+  promotion and performance reviews.
+* Organizations shall regularly report metrics of their open source
+  contributions and track these metrics over time.  These metrics may be
+  published only internally within the organization, or at the
+  organization’s discretion, some or all may be published externally.
+  Metrics that are strongly suggested include:
+
+  * The number of upstream kernel contributions by engineer, team, and
+    organization (e.g., all people reporting up to a manager, director,
+    or VP).
+  * The percentage of kernel developers who have made upstream
+    contributions relative to the total kernel developers in the
+    organization.
+  * The time interval between kernels used in the organization’s servers
+    and/or products, and the publication date of the upstream kernel
+    upon which the internal kernel is based.
+  * The number of out-of-tree commits present in internal kernels.
+
+Level 4
+=======
+
+* Software Engineers are encouraged to spend a portion of their work
+  time focused on Upstream Work, which is defined as reviewing patches
+  or papers, improving core project infrastructure such as writing or
+  maintaining tests, upstream tech debt reduction, writing
+  documentation, etc.
+* Software Engineers are supported in helping to organize Linux-related
+  conferences.
+* Organizations shall consider community member feedback in official
+  performance reviews.
+
+Level 5
+=======
+
+* Upstream kernel development is considered a formal job position, with
+  at least a third of the engineer’s time spent doing Upstream Work.
+* Organizations shall actively seek out community member feedback as a
+  factor in official performance reviews.
+* Organizations shall regularly report internally on the ratio of
+  Upstream Work to work focused on directly pursuing business goals.
diff --git a/Documentation/process/index.rst b/Documentation/process/index.rst
index d4b6217472b0..33715da7e684 100644
--- a/Documentation/process/index.rst
+++ b/Documentation/process/index.rst
@@ -50,6 +50,7 @@ Other guides to the community that are of interest to most developers are:
    embargoed-hardware-issues
    maintainers
    researcher-guidelines
+   contribution-maturity-model
 
 These are some overall technical guides that have been put here for now for
 lack of a better place.
-- 
2.31.0

Re: [PATCH] Documentation/process: Add Linux Kernel Contribution Maturity Model

[Michael S. Tsirkin]

On Wed, Dec 14, 2022 at 01:57:14PM -0500, Theodore Ts'o wrote:
> +  * The number of upstream kernel contributions by engineer, team, and
> +    organization (e.g., all people reporting up to a manager, director,
> +    or VP).

This is the only one that gave me pause. Seems eminently gameable by
trash variable-name-tweaking patches, unnecessary commit splitting etc.
I'm not going to name names but I'm seeing some of this as it is.
And if people decide to count patches posted as opposed to commits, oh
my ...  I don't think we should make people focus on the number of
contributions as opposed to their quality, and incorrect incentives tend
to promote incorrect behaviour.

Fundamentally I presume companies will evaluate contributions based on
how important the given functionality is for the company, so
I'm not sure we need to get involved here. Maybe just add this at the
end:

+    The metrics can be measured by team or
+    organization (e.g., all people reporting up to a manager, director,
+    or VP).

Or simply drop this part.
Besides this:

Acked-by: Michael S. Tsirkin <mst@redhat.com>


-- 
MST

Re: Documentation/process: Add Linux Kernel Contribution Maturity Model

[Linus Walleij]

> The goal is to encourage, in a management-friendly way, companies to
> allow their engineers to contribute with the upstream Linux Kernel
> development community, so we can grow the "talent pipeline" for
> contributors to become respected leaders, and eventually kernel
> maintainers.

These are noble goals. Also the bullet list is short, formal, and
to the point. This is nice.

I kind of side with Michael Tsirkin's point that quantitative
measures of performance can be harmful or give a false impression
of control. Consider this example:

linux$ git log --oneline --author=Walleij |wc -l
4301
linux$ git log --oneline --author=McKenney |wc -l
3346

There is something about some code in the kernel being more
"core" than others that needs to be taken into account, albeit
it is maybe an elusive concept. There is a hierarchy from
contributing syntactic changes to coding style to contributing
and maintaining say RCU.

That can be judged by peers and is addressed in other points in
the document so what about just dropping this paragraph?

Then there is this:

> +* Software Engineers are encouraged to spend a portion of their work
> +  time focused on Upstream Work, which is defined as reviewing patches
> +  or papers,

Papers! If we are linking ourselves to the Scientific Community
(and I see that as a good thing) we should certainly drop a few
words about it. It's a place of (ideally) meritocracy and
formal review of contributions.

Here is a real nice paper:
https://kernel.dk/systor13-final18.pdf
as a community we should probably produce more like this.

Or maybe you meant to say "mails". I don't know.

On a side note I would say that this document creates a really
serious cognitive dissonance with the document
Documentation/process/management-style.rst
so just for the fun of it I think this document should reference
that latter document as a guideline for how Software Engineers
will be evaluated in the community when they take on . It is a
nice nod to the nature of conflicting interests and worldviews
we need to deal with.

Yours,
Linus Walleij

Re: [PATCH] Documentation/process: Add Linux Kernel Contribution Maturity Model

[Bagas Sanjaya]

[-- Attachment #1: Type: text/plain, Size: 721 bytes --]

On Wed, Dec 14, 2022 at 01:57:14PM -0500, Theodore Ts'o wrote:
> +* Software Engineers are not allowed to contribute patches to the Linux
> +  kernel.

Software engineers turned product manager or support desk people?

> +  * The time interval between kernels used in the organization’s servers
> +    and/or products, and the publication date of the upstream kernel
> +    upon which the internal kernel is based.

I.e. the time for rebasing internal kernel against upstream one? For
example rebasing against several stable minor releases (x.y.z) is easier
than forward porting out-of-tree commits into new upstream release
(x.y).

Thanks.

-- 
An old man doll... just what I always wanted! - Clara

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

Re: Documentation/process: Add Linux Kernel Contribution Maturity Model

[Bagas Sanjaya]

[-- Attachment #1: Type: text/plain, Size: 1524 bytes --]

On Thu, Dec 15, 2022 at 04:14:36PM +0100, Linus Walleij wrote:
> > The goal is to encourage, in a management-friendly way, companies to
> > allow their engineers to contribute with the upstream Linux Kernel
> > development community, so we can grow the "talent pipeline" for
> > contributors to become respected leaders, and eventually kernel
> > maintainers.
> 
> These are noble goals. Also the bullet list is short, formal, and
> to the point. This is nice.
> 
> I kind of side with Michael Tsirkin's point that quantitative
> measures of performance can be harmful or give a false impression
> of control. Consider this example:
> 
> linux$ git log --oneline --author=Walleij |wc -l
> 4301
> linux$ git log --oneline --author=McKenney |wc -l
> 3346
> 
> There is something about some code in the kernel being more
> "core" than others that needs to be taken into account, albeit
> it is maybe an elusive concept. There is a hierarchy from
> contributing syntactic changes to coding style to contributing
> and maintaining say RCU.

Don't forget reputation produced as a result of contributions from the
company as a whole. For example, cdjrlc.com is known for noisy "redundant
word"-stripping patches and ignoring critical community reviews; ZTE
with "atypical" corporate email setup for which Greg dislike; and Intel
which (as you may know) has positive legacy to the community with its
high-quality contributions.

Thanks.

-- 
An old man doll... just what I always wanted! - Clara

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

Re: [PATCH] Documentation/process: Add Linux Kernel Contribution Maturity Model

[Theodore Ts'o]

On Fri, Dec 16, 2022 at 03:14:17PM +0700, Bagas Sanjaya wrote:
> On Wed, Dec 14, 2022 at 01:57:14PM -0500, Theodore Ts'o wrote:
> > +* Software Engineers are not allowed to contribute patches to the Linux
> > +  kernel.
> 
> Software engineers turned product manager or support desk people?

Actually, what we were thinking of companies that were paranoid about
the GPL.  For example, there is a particular major hardware supplier
used by many in the Linux ecosystem where the senior management tends
to be mostly composed of patent lawyers....

(In the United States, most companies have employment contracts which
means that all intellectual output, either on or off work.  Which
means that some companies can prohibit their employees from
contributing anything to any open source project, even if the work is
done after hours on their own time.  So while "Level 0" may seem
horrific, there are software engineers in the US who are constrained
in this way.  If we can encourage those companies to move from Level 0
to Level 1, that's at least a step in the right direction.)

> > +  * The time interval between kernels used in the organization’s servers
> > +    and/or products, and the publication date of the upstream kernel
> > +    upon which the internal kernel is based.
> 
> I.e. the time for rebasing internal kernel against upstream one? For
> example rebasing against several stable minor releases (x.y.z) is easier
> than forward porting out-of-tree commits into new upstream release
> (x.y).

It's not necessarily the time to rebase the kernel.  It might take
less than, say, six years to rebase a product kernel based on 4.9 to
6.1.  But the company might decide not to expend the engineering
resources to do the work, but instead to continue shipping product or
using a data center kernel based on an antediluvian upstream release.
After all, there are many companies using RHEL/CentOS 7, whose kernel
is based on 3.10.  Yeah, with of tons of backports and (hopefully) all
of the necessary security patches, but is that really the best way to
run a railroad?

But, yes, if your point is that this metric doesn't distinguish
between an organization which is regularly keeping itself with the
very latest 4.9.336 kernel, as opposed to reguarly jumping to the most
recent LTS kernel (i.e., every year or every other years going from
5.4 to 5.10 to 5.15 to 6.1), yes that's true.

However, it may be that it's better to encourage companies to adopt an
"upstream-first" development model.  I attempted to capture this in
the metric, "number of out-of-tree commits present in internal
kernels".

						- Ted

Re: [PATCH] Documentation/process: Add Linux Kernel Contribution Maturity Model

[Florian Fainelli]

[-- Attachment #1: Type: text/plain, Size: 7595 bytes --]

Hello,

On 12/14/2022 10:57 AM, Theodore Ts'o wrote:
> As a follow-up to a discussion at the 2021 Maintainer's Summit on the
> topic of maintainer recruitment and retention, the TAB took on the
> task of creating a document which to help companies and other
> organizations to grow in their ability to engage with the Linux Kernel
> development community, using the Maturity Model[2] framework.
> 
> The goal is to encourage, in a management-friendly way, companies to
> allow their engineers to contribute with the upstream Linux Kernel
> development community, so we can grow the "talent pipeline" for
> contributors to become respected leaders, and eventually kernel
> maintainers.
> 
> [1] https://lwn.net/Articles/870581/
> [2] https://en.wikipedia.org/wiki/Maturity_model
> 
> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
> Co-developed-by: Kees Cook <keescook@chromium.org>
> Signed-off-by: Kees Cook <keescook@chromium.org>
> Co-developed-by: Dan Williams <dan.j.williams@intel.com>
> Signed-off-by: Dan Williams <dan.j.williams@intel.com>
> Acked-by: Jakub Kicinski <kuba@kernel.org>
> Acked-by: Christian Brauner (Microsoft) <brauner@kernel.org>
> Acked-by: Dave Hansen <dave.hansen@linux.intel.com>
> ---

[snip]

> +The TAB urges organizations to continuously evaluate their Open Source
> +maturity model and commit to improvements to align with this model.  To
> +be effective, this evaluation should incorporate feedback from across
> +the organization, including management and developers at all seniority
> +levels.  In the spirit of Open Source, we encourage organizations to
> +publish their evaluations and plans to improve their engagement with the
> +upstream community.

You probably need to provide a word or two about who is the intended 
recipient of this document, whether this is intended to be utilized by 
software developers, their management chain, or both.

> +
> +Level 0
> +=======
> +
> +* Software Engineers are not allowed to contribute patches to the Linux
> +  kernel.
> +
> +
> +Level 1
> +=======
> +
> +* Software Engineers are allowed to contribute patches to the Linux
> +  kernel, either as part of their job responsibilities or on their own
> +  time.
> +
> +Level 2
> +=======
> +
> +* Software Engineers are expected to contribute to the Linux Kernel as
> +  part of their job responsibilities.
> +* Software Engineers will be supported to attend Linux-related
> +  conferences as a part of their job.

This is not something IMHO that qualifies for Level 2, but rather for 
Level 3. It is reasonably simple and easy for companies to support their 
engineers contributing to the kernel, it is a whole different thing to 
financially support them to attend conference(s).

> +* A Software Engineer’s upstream code contributions shall be considered
> +  in promotion and performance reviews.
> +
> +Level 3
> +=======
> +
> +* Software Engineers are expected to review patches (including patches
> +  authored by engineers from other companies) as part of their job
> +  responsibilities

Would substitute expected with highly encouraged here.

> +* Contributing presentations or papers to Linux-related conferences are
> +  considered part of an engineer’s work.

Again, that seems more of a Level 4 item to me.

> +* A Software Engineer’s community contributions shall be considered in
> +  promotion and performance reviews.
> +* Organizations shall regularly report metrics of their open source
> +  contributions and track these metrics over time.  These metrics may be
> +  published only internally within the organization, or at the
> +  organization’s discretion, some or all may be published externally.
> +  Metrics that are strongly suggested include:
> +
> +  * The number of upstream kernel contributions by engineer, team, and
> +    organization (e.g., all people reporting up to a manager, director,
> +    or VP).
> +  * The percentage of kernel developers who have made upstream
> +    contributions relative to the total kernel developers in the
> +    organization.
> +  * The time interval between kernels used in the organization’s servers
> +    and/or products, and the publication date of the upstream kernel
> +    upon which the internal kernel is based.

That part is a bit difficult to generalize, so as a broad definition of 
metrics to track, it seems appropriate as a possible example. In 
practice lots of people have to work within the bounds of the kernel 
they are provided with, and they may not always entirely control the 
contribution process behind as much as the code getting in. It seems to 
me that we have usually following categories:

- everything lands upstream and/or working upstream first which is the 
case with Intel, AMD, Meta, Oracle, etc.

- maintaining a major distribution kernel: Android, RedHat, SUSE, Oracle 
UKE with as much upstream code downstream as downstream code reaching 
upstream eventually

- working with a well maintained downstream kernel tree based upon a LTS 
version from a vendor where most SoCs companies fall these days hopefully

- working with a not so major distribution kernel in the likes of 
OpenWrt etc.

- unmaintained kernel


> +  * The number of out-of-tree commits present in internal kernels.

The metrics that an organization must provide versus the level a given 
engineer is at is independent from one another, or rather, you may want 
to create different levels, one for engineers which is what you have, 
and one for organizations.

So maybe there needs to be 4 levels for an organization:

0 - no support/awareness/will to engage
1 - mild engagement with the community in one form of another, including 
maintenance of metrics
2 - organization is largely supportive of upstream work and does so with 
pouring some occasional resources to attend/organize conferences, albeit 
not at the "gold" sponsor level
3 - organization heavily leans on upstream work for its products and is 
a regular sponsor at major events

> +
> +Level 4
> +=======
> +
> +* Software Engineers are encouraged to spend a portion of their work
> +  time focused on Upstream Work, which is defined as reviewing patches
> +  or papers, improving core project infrastructure such as writing or
> +  maintaining tests, upstream tech debt reduction, writing
> +  documentation, etc.
> +* Software Engineers are supported in helping to organize Linux-related
> +  conferences.

Likewise, this seems to be more of a Level 5 aspect rather than Level 4.

> +* Organizations shall consider community member feedback in official
> +  performance reviews.
> +
> +Level 5

And I would make this essentially Level 6.

> +=======
> +
> +* Upstream kernel development is considered a formal job position, with
> +  at least a third of the engineer’s time spent doing Upstream Work.
> +* Organizations shall actively seek out community member feedback as a
> +  factor in official performance reviews.
> +* Organizations shall regularly report internally on the ratio of
> +  Upstream Work to work focused on directly pursuing business goals.
> diff --git a/Documentation/process/index.rst b/Documentation/process/index.rst
> index d4b6217472b0..33715da7e684 100644
> --- a/Documentation/process/index.rst
> +++ b/Documentation/process/index.rst
> @@ -50,6 +50,7 @@ Other guides to the community that are of interest to most developers are:
>      embargoed-hardware-issues
>      maintainers
>      researcher-guidelines
> +   contribution-maturity-model
>   
>   These are some overall technical guides that have been put here for now for
>   lack of a better place.

-- 
Florian

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4221 bytes --]

Re: [PATCH] Documentation/process: Add Linux Kernel Contribution Maturity Model

[Jonathan Corbet]

"Theodore Ts'o" <tytso@mit.edu> writes:

> As a follow-up to a discussion at the 2021 Maintainer's Summit on the
> topic of maintainer recruitment and retention, the TAB took on the
> task of creating a document which to help companies and other
> organizations to grow in their ability to engage with the Linux Kernel
> development community, using the Maturity Model[2] framework.
>
> The goal is to encourage, in a management-friendly way, companies to
> allow their engineers to contribute with the upstream Linux Kernel
> development community, so we can grow the "talent pipeline" for
> contributors to become respected leaders, and eventually kernel
> maintainers.
>
> [1] https://lwn.net/Articles/870581/
> [2] https://en.wikipedia.org/wiki/Maturity_model

So there were a few review comments on this document...  Ted, were you
planning a new revision or is it your feeling that it's ready to be
merged in this form?

Thanks,

jon