[PATCH] Documentation: Linux Contribution Maturity Model and the wider community

[PATCH] Documentation: Linux Contribution Maturity Model and the wider community

[Finn Thain]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 2585 bytes --]

The Linux Contribution Maturity Model methodology is notionally based on
the Open source Maturity Model (OMM) which was in turn based on the
Capability Maturity Model Integration (CMMI).

According to Petrinja et al., the goal of the OMM was to extend the CMMI
so as to be useful both for companies and for communities [1][2]. However,
the Linux Contribution Maturity Model considers only companies and
businesses.

This patch addresses this bias as it could hinder collaboration with
not-for-profit organisations and individuals, which would be a loss to
any stakeholder.

Level 5 is amended to remove the invitation to exercise the same bias
i.e. employees rewarded indirectly by other companies.

[1] Petrinja, E., Nambakam, R., Sillitti, A.: Introducing the
OpenSource Maturity Model. In: 2nd Emerging Trends in FLOSS Research
and Development Workshop at ICSE 2009, Vancouver, BC, Canada (2009)

[2] Wittmann, M., Nambakam, R.: Qualipso Deliverable A6.D1.6.3
CMM-like model for OSS.

Cc: Theodore Ts'o <tytso@mit.edu>
Cc: Kees Cook <keescook@chromium.org>
Cc: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Finn Thain <fthain@linux-m68k.org>
---
 Documentation/process/contribution-maturity-model.rst | 9 ++++-----
 1 file changed, 4 insertions(+), 5 deletions(-)

diff --git a/Documentation/process/contribution-maturity-model.rst b/Documentation/process/contribution-maturity-model.rst
index b87ab34de22c..863a2e4c22e2 100644
--- a/Documentation/process/contribution-maturity-model.rst
+++ b/Documentation/process/contribution-maturity-model.rst
@@ -62,8 +62,8 @@ Level 3
 =======
 
 * Software Engineers are expected to review patches (including patches
-  authored by engineers from other companies) as part of their job
-  responsibilities
+  authored by contributors from outside of the organization) as part of
+  their job responsibilities
 * Contributing presentations or papers to Linux-related or academic
   conferences (such those organized by the Linux Foundation, Usenix,
   ACM, etc.), are considered part of an engineer’s work.
@@ -103,7 +103,6 @@ Level 5
 
 * Upstream kernel development is considered a formal job position, with
   at least a third of the engineer’s time spent doing Upstream Work.
-* Organizations will actively seek out community member feedback as a
-  factor in official performance reviews.
 * Organizations will regularly report internally on the ratio of
-  Upstream Work to work focused on directly pursuing business goals.
+  Upstream Work to work focused on directly pursuing the organisation's
+  other goals.
-- 
2.39.3

Re: [PATCH] Documentation: Linux Contribution Maturity Model and the wider community

[Greg Kroah-Hartman]

On Mon, Jun 19, 2023 at 07:41:57PM +1000, Finn Thain wrote:
> The Linux Contribution Maturity Model methodology is notionally based on
> the Open source Maturity Model (OMM) which was in turn based on the
> Capability Maturity Model Integration (CMMI).
> 
> According to Petrinja et al., the goal of the OMM was to extend the CMMI
> so as to be useful both for companies and for communities [1][2]. However,
> the Linux Contribution Maturity Model considers only companies and
> businesses.
> 
> This patch addresses this bias as it could hinder collaboration with
> not-for-profit organisations and individuals, which would be a loss to
> any stakeholder.
> 
> Level 5 is amended to remove the invitation to exercise the same bias
> i.e. employees rewarded indirectly by other companies.
> 
> [1] Petrinja, E., Nambakam, R., Sillitti, A.: Introducing the
> OpenSource Maturity Model. In: 2nd Emerging Trends in FLOSS Research
> and Development Workshop at ICSE 2009, Vancouver, BC, Canada (2009)
> 
> [2] Wittmann, M., Nambakam, R.: Qualipso Deliverable A6.D1.6.3
> CMM-like model for OSS.
> 
> Cc: Theodore Ts'o <tytso@mit.edu>
> Cc: Kees Cook <keescook@chromium.org>
> Cc: Dan Williams <dan.j.williams@intel.com>
> Signed-off-by: Finn Thain <fthain@linux-m68k.org>
> ---
>  Documentation/process/contribution-maturity-model.rst | 9 ++++-----
>  1 file changed, 4 insertions(+), 5 deletions(-)
> 
> diff --git a/Documentation/process/contribution-maturity-model.rst b/Documentation/process/contribution-maturity-model.rst
> index b87ab34de22c..863a2e4c22e2 100644
> --- a/Documentation/process/contribution-maturity-model.rst
> +++ b/Documentation/process/contribution-maturity-model.rst
> @@ -62,8 +62,8 @@ Level 3
>  =======
>  
>  * Software Engineers are expected to review patches (including patches
> -  authored by engineers from other companies) as part of their job
> -  responsibilities
> +  authored by contributors from outside of the organization) as part of
> +  their job responsibilities

This is fine, but:

>  * Contributing presentations or papers to Linux-related or academic
>    conferences (such those organized by the Linux Foundation, Usenix,
>    ACM, etc.), are considered part of an engineer’s work.
> @@ -103,7 +103,6 @@ Level 5
>  
>  * Upstream kernel development is considered a formal job position, with
>    at least a third of the engineer’s time spent doing Upstream Work.
> -* Organizations will actively seek out community member feedback as a
> -  factor in official performance reviews.

Why are you removing this?  I write more performance reviews now than I
have have in my life, all for companies that I do NOT work for.  That's
a good thing as it shows these orginizations value the feedback of the
community as a reflection on how well those employees are doing at their
assigned job.  Why are you removing that very valid thing?

>  * Organizations will regularly report internally on the ratio of
> -  Upstream Work to work focused on directly pursuing business goals.
> +  Upstream Work to work focused on directly pursuing the organisation's

This is a good change.

thanks,

greg k-h

Re: [PATCH] Documentation: Linux Contribution Maturity Model and the wider community

[Finn Thain]

[-- Attachment #1: Type: text/plain, Size: 2220 bytes --]


On Mon, 19 Jun 2023, Greg Kroah-Hartman wrote:

> On Mon, Jun 19, 2023 at 07:41:57PM +1000, Finn Thain wrote:

> > @@ -103,7 +103,6 @@ Level 5
> >  
> >  * Upstream kernel development is considered a formal job position, with
> >    at least a third of the engineer’s time spent doing Upstream Work.
> > -* Organizations will actively seek out community member feedback as a
> > -  factor in official performance reviews.
> 
> Why are you removing this?  I write more performance reviews now than I 
> have have in my life, all for companies that I do NOT work for.  That's 
> a good thing as it shows these orginizations value the feedback of the 
> community as a reflection on how well those employees are doing at their 
> assigned job.  Why are you removing that very valid thing?
> 

I'm not preventing that. That's covered by level 4 and my patch only 
alters level 3 and level 5.

Bonuses and salaries are tied to performance reviews so the hazard here 
are clear. Level 5 compels companies to seek feedback and naturally they 
will seek it from companies who share their goals. You ask too much of 
employees if you expect them to put aside the corporate agendas and pursue 
the interests of the wider community.

Countless lawsuits over the last few decades made it abundantly clear that 
the goals of companies often diverge from those of the wider FLOSS 
community.

Consider all of the open source code thrown over the wall, the binary 
blobs, the binary modules, the built-in obsolescence, the devices shipped 
with vulnerabilities now reduced to e-waste because they cannot be fixed, 
the vendor lock-in strategies, the walled gardens, the surveillance etc.

To my jaded mind, it is obvious that such reprehensible strategies can be 
advanced by co-operative employees given inducements from colluding 
companies. My patch won't prevent this sort of behaviour but it does 
remove a directive that would help facilitate it.

Greg, if you want to see more performance reviews, the maturity model 
could compel companies to provide unsolicited feedback, instead of seek it 
from an arbitrary source. Would you be amenable to a revised patch along 
those lines?

Re: [PATCH] Documentation: Linux Contribution Maturity Model and the wider community

[James Bottomley]

On Tue, 2023-06-20 at 13:48 +1000, Finn Thain wrote:
> 
> On Mon, 19 Jun 2023, Greg Kroah-Hartman wrote:
> 
> > On Mon, Jun 19, 2023 at 07:41:57PM +1000, Finn Thain wrote:
> 
> > > @@ -103,7 +103,6 @@ Level 5
> > >  
> > >  * Upstream kernel development is considered a formal job
> > > position, with
> > >    at least a third of the engineer’s time spent doing Upstream
> > > Work.
> > > -* Organizations will actively seek out community member feedback
> > > as a
> > > -  factor in official performance reviews.
> > 
> > Why are you removing this?  I write more performance reviews now
> > than I have have in my life, all for companies that I do NOT work
> > for. That's a good thing as it shows these orginizations value the
> > feedback of the  community as a reflection on how well those
> > employees are doing at their assigned job.  Why are you removing
> > that very valid thing?
> > 
> 
> I'm not preventing that. That's covered by level 4 and my patch only 
> alters level 3 and level 5.
> 
> Bonuses and salaries are tied to performance reviews so the hazard
> here are clear. Level 5 compels companies to seek feedback and
> naturally they will seek it from companies who share their goals. You
> ask too much of  employees if you expect them to put aside the
> corporate agendas and pursue the interests of the wider community.

Actually, I don't think we are.  Part of the mechanical effects of the
open source revolution was to empower employees over employers: it's
the employees who submit the code and are part of the community, not
the employer.  In many ways employees in Open Source become Ambassadors
and Agents for their employers.  There's a big drive in Foundation
driven Corporate Open Source to try to minimize this employee
empowerement effect, but it's there non the less.  A good open source
employee recognizes this, often moves employers keeping the same open
source community roles and tries to find a synergy between corporate
goals and community ones (the best actually alter the corporate goals
to effect this).

> Countless lawsuits over the last few decades made it abundantly clear
> that the goals of companies often diverge from those of the wider
> FLOSS community.

Yes, but with good employee guidance, convergence can be found.  In
many ways community manager positions at companies are about managing
the company goals rather than the community ...

> Consider all of the open source code thrown over the wall, the binary
> blobs, the binary modules, the built-in obsolescence, the devices
> shipped with vulnerabilities now reduced to e-waste because they
> cannot be fixed,  the vendor lock-in strategies, the walled gardens,
> the surveillance etc.

It's employers' money and time if they want to waste it in this
fashion.  Unfortunately theoretical education isn't always the answer
and some entities need a burned hand as a teacher.

> To my jaded mind, it is obvious that such reprehensible strategies
> can be advanced by co-operative employees given inducements from
> colluding companies. My patch won't prevent this sort of behaviour
> but it does remove a directive that would help facilitate it.

Most things in life can be abused.  When stating something like this
we're trying to encourage people to listen to their better angels even
if it risks abuse.


James

Re: [PATCH] Documentation: Linux Contribution Maturity Model and the wider community

[James Bottomley]

On Mon, Jun 19, 2023 at 07:41:57PM +1000, Finn Thain wrote:
> The Linux Contribution Maturity Model methodology is notionally based
> on the Open source Maturity Model (OMM) which was in turn based on
> the Capability Maturity Model Integration (CMMI).
> 
> According to Petrinja et al., the goal of the OMM was to extend the
> CMMI so as to be useful both for companies and for communities
> [1][2].  However, the Linux Contribution Maturity Model considers
> only companies and businesses.

That's not a correct characterization.  The model is designed to
measure and be useful to businesses, but it definitely considers the
community because it's progress is built around being more useful to
and working more effectively with the community.

> This patch addresses this bias as it could hinder collaboration with
> not-for-profit organisations and individuals, which would be a loss
> to any stakeholder.

I don't really think changing 'Businesses' to 'Organizations' entirely
addresses what you claim is the bias because individuals would still be
excluded from the term 'Organizations'.  I also don't really think it
matters.  Part of the reason this whole thing doesn't matter is that
sometimes people do know who a contributor they work with works for,
but most of the time they don't.  If you really want this to be
inclusive, you could change it to 'other contributors' but I'm still
not sure it's worth it.

> 
> Level 5 is amended to remove the invitation to exercise the same bias
> i.e. employees rewarded indirectly by other companies.

I also wouldn't remove the bit about seeking upstream feedback on
employees; I know from personal experience it happens a lot.

Regards,

James

Re: [PATCH] Documentation: Linux Contribution Maturity Model and the wider community

[Finn Thain]

On Mon, 19 Jun 2023, James Bottomley wrote:

> On Mon, Jun 19, 2023 at 07:41:57PM +1000, Finn Thain wrote:
> > The Linux Contribution Maturity Model methodology is notionally based 
> > on the Open source Maturity Model (OMM) which was in turn based on the 
> > Capability Maturity Model Integration (CMMI).
> > 
> > According to Petrinja et al., the goal of the OMM was to extend the 
> > CMMI so as to be useful both for companies and for communities [1][2].  
> > However, the Linux Contribution Maturity Model considers only 
> > companies and businesses.
> 
> That's not a correct characterization.  The model is designed to measure 
> and be useful to businesses, but it definitely considers the community 
> because it's progress is built around being more useful to and working 
> more effectively with the community.
> 

You're right, the characterization I gave does exaggerate the bias. I 
shall moderate that if I resubmit the patch.

> > This patch addresses this bias as it could hinder collaboration with 
> > not-for-profit organisations and individuals, which would be a loss to 
> > any stakeholder.
> 
> I don't really think changing 'Businesses' to 'Organizations' entirely 
> addresses what you claim is the bias because individuals would still be 
> excluded from the term 'Organizations'.  I also don't really think it 
> matters.  Part of the reason this whole thing doesn't matter is that 
> sometimes people do know who a contributor they work with works for, but 
> most of the time they don't.

This is not just about patches, it's also about incentives and influence.

> If you really want this to be inclusive, you could change it to 'other 
> contributors' but I'm still not sure it's worth it.
> 
> > 
> > Level 5 is amended to remove the invitation to exercise the same bias
> > i.e. employees rewarded indirectly by other companies.
> 
> I also wouldn't remove the bit about seeking upstream feedback on
> employees; I know from personal experience it happens a lot.
> 

If it happens a lot already, why compel employers to seek it?

It's worth noting that the model compels employers to seek "community 
member feedback" which is not the same as the "upstream feedback" that you 
describe.

Re: [Tech-board-discuss] [PATCH] Documentation: Linux Contribution Maturity Model and the wider community

[James Bottomley]

On Tue, 2023-06-20 at 13:50 +1000, Finn Thain wrote:
> On Mon, 19 Jun 2023, James Bottomley wrote:
> 
> > On Mon, Jun 19, 2023 at 07:41:57PM +1000, Finn Thain wrote:
> > > The Linux Contribution Maturity Model methodology is notionally
> > > based on the Open source Maturity Model (OMM) which was in turn
> > > based on the Capability Maturity Model Integration (CMMI).
> > > 
> > > According to Petrinja et al., the goal of the OMM was to extend
> > > the  CMMI so as to be useful both for companies and for
> > > communities [1][2].   However, the Linux Contribution Maturity
> > > Model considers only companies and businesses.
> > 
> > That's not a correct characterization.  The model is designed to
> > measure and be useful to businesses, but it definitely considers
> > the community because it's progress is built around being more
> > useful to and working more effectively with the community.
> > 
> 
> You're right, the characterization I gave does exaggerate the bias. I
> shall moderate that if I resubmit the patch.
> 
> > > This patch addresses this bias as it could hinder collaboration
> > > with  not-for-profit organisations and individuals, which would
> > > be a loss to  any stakeholder.
> > 
> > I don't really think changing 'Businesses' to 'Organizations'
> > entirely addresses what you claim is the bias because individuals
> > would still be excluded from the term 'Organizations'.  I also
> > don't really think it matters.  Part of the reason this whole thing
> > doesn't matter is that sometimes people do know who a contributor
> > they work with works for, but most of the time they don't.
> 
> This is not just about patches, it's also about incentives and
> influence.

I mentioned contributor interaction, which covers influence.  I'm not
sure what you mean by incentives or how it is covered by changing
Businesses -> Organizations.

> 
> > If you really want this to be inclusive, you could change it to
> > 'other contributors' but I'm still not sure it's worth it.
> > 
> > > 
> > > Level 5 is amended to remove the invitation to exercise the same
> > > bias i.e. employees rewarded indirectly by other companies.
> > 
> > I also wouldn't remove the bit about seeking upstream feedback on
> > employees; I know from personal experience it happens a lot.
> > 
> 
> If it happens a lot already, why compel employers to seek it?

Because it's a sign of open source maturity on behalf of a company. 
Lots do it, but lots don't.  By putting it in the maturity model we
want to encourage it.

> It's worth noting that the model compels employers to seek "community
> member feedback" which is not the same as the "upstream feedback"
> that you describe.

It isn't?  How else does a community express itself except by its
agents which are ipso facto community members?  Not all community
members have identical opinions, but if you talk to several you'll get
a good sense of community feedback.

James

Re: [PATCH] Documentation: Linux Contribution Maturity Model and the wider community

[Theodore Ts'o]

On Mon, Jun 19, 2023 at 07:41:57PM +1000, Finn Thain wrote:
> The Linux Contribution Maturity Model methodology is notionally based on
> the Open source Maturity Model (OMM) which was in turn based on the
> Capability Maturity Model Integration (CMMI).

So from a historical/factual basis, this is not true.  It was *not*
based on the Open Source Maturity Model; in fact, as the principal
author of this document, I wasn't even aware of the OMM when I wrote
it.  The general framework of Maturity Models[1] is a very long one, and
goes back well beyond Dececmber 2022, which is when the folks in the
banking sector launched the OMM[2].

[1] https://en.wikipedia.org/wiki/Maturity_model
[2] https://www.finos.org/blog/open-source-maturity-model-launch

The reason why the language in the Linux Contribution Maturity Model
(LCMM) focused on companies was that was where the problem was
perceived to be.  That is, the *vast* majority of Linux Kernel
contributors work at companies, and because of many companys' focus on
reducing costs and increasing profits of their products which are part
of the Linux kernel ecosystem, some of them enagage in anti-patterns
which are not healthy either for their own role in the Linux Kernel
ecosystem, and for the Linux Kernel ecosystem at large.

For example, if you look at the 6.3 contribution report[3], you'll see
that by changesets (git commits), 85.4% of the contributions came from
companies, 6.6% were unknown, 4.8% were "None" (e.g.,
hobbists/students), and 1.1% were from the Linux Foundation.

[3] https://lwn.net/Articles/929582/

In actual practice, we get *very* few commits from non-profit
organizations such as, say, the Mozilla Foundation, the Eclipse
Foundation, or even community distributions such as Debian or Arch.
And so the concerns around software engineers not getting the
permission and encourage they need so they can contribute to the Linux
kernel community at large, is primarily coming from companies.  The
only non-profit organization that even shows up at the contribution
reports is the Linux Foundation, and I'm pretty confident in how
enlightened the LF management vis-a-vis kernel contribution.  :-)

As far as individuals are concerned, things like performance reviews,
the ability for overly paranoid corporate Corporate General Counsel
not allowing their engineers from contributing to Open Source (in
general) and the Linux Kernel (in particular), yes, those things
aren't really applicable.  But again, there is a specific problem that
we are trying to solve, and it's not with individual contriduals.

> This patch addresses this bias as it could hinder collaboration with
> not-for-profit organisations and individuals, which would be a loss to
> any stakeholder.

I'm not sure how this document would "hinder collaboration" with
non-profit organizations and individuals.  Could you say more about
your concern regarding how this undesireable outcome would happen in
practice?

I'm not against making using wording which is more general, such as
perhaps "companies and organizations" instead of "companies", but it's
important that we don't dilute the message the primary audience ---
which is pointed-haired management types, who are familiar with the
Maturity Model framework, who are *primarily* working at for-profit
companies, and who could make it easier for those Linux developers
whose day job involves Linux kernel development.

Cheers,

						- Ted

Re: [PATCH] Documentation: Linux Contribution Maturity Model and the wider community

[Finn Thain]

On Mon, 19 Jun 2023, Theodore Ts'o wrote:

> On Mon, Jun 19, 2023 at 07:41:57PM +1000, Finn Thain wrote:
> > The Linux Contribution Maturity Model methodology is notionally based 
> > on the Open source Maturity Model (OMM) which was in turn based on the 
> > Capability Maturity Model Integration (CMMI).
> 
> So from a historical/factual basis, this is not true.  It was *not* 
> based on the Open Source Maturity Model; in fact, as the principal 
> author of this document, I wasn't even aware of the OMM when I wrote it.  
> The general framework of Maturity Models[1] is a very long one, and goes 
> back well beyond Dececmber 2022, which is when the folks in the banking 
> sector launched the OMM[2].
> 
> [1] https://en.wikipedia.org/wiki/Maturity_model
> [2] https://www.finos.org/blog/open-source-maturity-model-launch
> 

Thanks for that background, and thanks for your work on the Linux model. I 
appreciate the basic aim of the Linux model though I am highly skeptical 
of the relevance of a top-down goal-oriented methodology to a bottom-up 
compromise-oriented collaborative effort.

With regard to that mismatch, though somewhat off-topic, I'll note that 
the Linux model has departed quite a distance from the CMMI. E.g. CMMI 
level 5 is about continuous improvement, whereas Linux level 5 is 
basically level 4 "only louder". In the CMMI, the elements that make up 
the lower levels are strictly required by those of the upper levels; so 
it's not a question of degree but necessity.

> The reason why the language in the Linux Contribution Maturity Model 
> (LCMM) focused on companies was that was where the problem was perceived 
> to be.  That is, the *vast* majority of Linux Kernel contributors work 
> at companies, and because of many companys' focus on reducing costs and 
> increasing profits of their products which are part of the Linux kernel 
> ecosystem, some of them enagage in anti-patterns which are not healthy 
> either for their own role in the Linux Kernel ecosystem, and for the 
> Linux Kernel ecosystem at large.
> 
> For example, if you look at the 6.3 contribution report[3], you'll see
> that by changesets (git commits), 85.4% of the contributions came from
> companies, 6.6% were unknown, 4.8% were "None" (e.g.,
> hobbists/students), and 1.1% were from the Linux Foundation.
> 
> [3] https://lwn.net/Articles/929582/
> 
> In actual practice, we get *very* few commits from non-profit 
> organizations such as, say, the Mozilla Foundation, the Eclipse 
> Foundation, or even community distributions such as Debian or Arch. And 
> so the concerns around software engineers not getting the permission and 
> encourage they need so they can contribute to the Linux kernel community 
> at large, is primarily coming from companies.  The only non-profit 
> organization that even shows up at the contribution reports is the Linux 
> Foundation, and I'm pretty confident in how enlightened the LF 
> management vis-a-vis kernel contribution.  :-)
> 

I suspect that counting commits may be the wrong metric (I can think of 
better ones). But if that's what we have, the lack of commits from 
non-profit organizations is a situation that might actually be improved by 
changes like the ones I'm advocating.

> As far as individuals are concerned, things like performance reviews, 
> the ability for overly paranoid corporate Corporate General Counsel not 
> allowing their engineers from contributing to Open Source (in general) 
> and the Linux Kernel (in particular), yes, those things aren't really 
> applicable.  But again, there is a specific problem that we are trying 
> to solve, and it's not with individual contriduals.
> 
> > This patch addresses this bias as it could hinder collaboration with 
> > not-for-profit organisations and individuals, which would be a loss to 
> > any stakeholder.
> 
> I'm not sure how this document would "hinder collaboration" with 
> non-profit organizations and individuals.  Could you say more about your 
> concern regarding how this undesireable outcome would happen in 
> practice?
> 

I believe that I've now addressed this in my message to Greg.

> I'm not against making using wording which is more general, such as 
> perhaps "companies and organizations" instead of "companies", but it's 
> important that we don't dilute the message the primary audience --- 
> which is pointed-haired management types, who are familiar with the 
> Maturity Model framework, who are *primarily* working at for-profit 
> companies, and who could make it easier for those Linux developers whose 
> day job involves Linux kernel development.
> 

If employers are going to make those day jobs easier, IMHO it will be quid 
pro quo or not at all. That's why I am wary of bias.

Re: [PATCH] Documentation: Linux Contribution Maturity Model and the wider community

[Theodore Ts'o]

On Tue, Jun 20, 2023 at 01:54:23PM +1000, Finn Thain wrote:
> 
> I suspect that counting commits may be the wrong metric (I can think of 
> better ones).

As far as whether counting commits is the wrong metric, it doesn't
matter if you look at lines of code changes, attendance at conferences
such as the Linux Plumbers Conference, or participation on weekly
subsystem calls or video conferences, it's fair to say that that the
vast majority of those current developers work for companies. 

> But if that's what we have, the lack of commits from 
> non-profit organizations is a situation that might actually be improved by 
> changes like the ones I'm advocating.

Not only have you not provided any evidence for your thesis (which to
be fair is hard since what you are positing is a hypothetical), but
you haven't even provided a theory why you believe that to be true!

I *do* have a theory for what we've observed, which is that developers
like to have food with their meals, and the amount of time and effort
to do serious kernel work is significant.  And so companies are the
ones who can fund the engineers which are spending a good proportion
of their time working on Linux kerenl development.  These is why we
see a huge amount of work from people who work for Linaro, Intel, Red
Hat, Google, IBM, Nvidia, Facebook, Oracle, etc. both in terms of code
contributed to the kernel, code reviews of other people's code
submission, design discussions on LKML and in various "hallway track
conversations" at the verious Linux conferences/workshops, etc.

As far as non-profit organizations are concerned, most of them have
very tightly defined mission.  That mission might be mostly unrelated
to the kernel, except as a customer of the kernel --- for example, the
Mozilla and Eclipse Foundation --- or it might be Linux adjacent, such
as "make a good distributions ala Debian, Arch, Fedora, etc."  The
people at these non-profits might be volunteers (this is mostly the
case for Debian), or they might be paid engineers based on the
corporate sponsoprs of the non-profits (for example) or engineers who
are paid by a company, but who are "on loan" to the non-profit
organization (for example, in the Apache Software Foundation this is a
common pattern).  Either way, though, those non-profits tend to have a
very tightly focused mission, and it tends not to be one which
requires a large amount of kernel development.

You appear to have a very different model of how non-profits might
approach the Linux kernel --- could you go into more detail about why
they might want to contribute to the Linux kernel, and how we might
encourage them to contribute more engineering effort?

> > I'm not sure how this document would "hinder collaboration" with 
> > non-profit organizations and individuals.  Could you say more about your 
> > concern regarding how this undesireable outcome would happen in 
> > practice?
> 
> I believe that I've now addressed this in my message to Greg.

Well, no, you haven't.   More below....

On Tue, Jun 20, 2023 at 01:48:59PM +1000, Finn Thain wrote: (in reply to Greg)
> 
> Bonuses and salaries are tied to performance reviews so the hazard here 
> are clear. Level 5 compels companies to seek feedback and naturally they 
> will seek it from companies who share their goals. You ask too much of 
> employees if you expect them to put aside the corporate agendas and pursue 
> the interests of the wider community.

I was a hobbyist from 1991 to 1999 (I was the first North American
linux kernel developer, and at the time my day job was tech lead for
the MIT KerBeros Team and I also served on the IETF Security Area
Directorate and was one of the IPSEC working group chairs), and then
from 1999 until present, I've worked for companies (first VA Linux
Systems, then the IBM Linux Technology Cetner, and now at Google).  So
I think I know something about how employees balance the needs of the
Linux Kernel community and the needs of their employer.

> Countless lawsuits over the last few decades made it abundantly clear that 
> the goals of companies often diverge from those of the wider FLOSS 
> community.
> 
> Consider all of the open source code thrown over the wall, the binary 
> blobs, the binary modules, the built-in obsolescence, the devices shipped 
> with vulnerabilities now reduced to e-waste because they cannot be fixed, 
> the vendor lock-in strategies, the walled gardens, the surveillance etc.

There haven't been *that* many lawsuits, and while there have been
some bad actors, there have also been many, MANY examples of companies
that have contributed in highly positive ways.  For example, well over
a decade ago, IBM started requiring that their peripheral card
suppliers (e.g., network cards, SCSI host bus adapters, etc.) that it
would be a requirement that thosse companies providing those
peripherals MUST have upstream Linux device drivers as a condition of
the procurement contract.

And the more enlightened companies *do* understood that out-of-tree
patches are technical debt, and to get drivers, patches, etc.,
upstream in the long run would save them huge amounts of effort.  So
there are plenty of ways in which the meeting the goals of the FLOSS
comunity is ultimately, good towards achieving the goals of for-profit
companies.

> To my jaded mind, it is obvious that such reprehensible strategies can be 
> advanced by co-operative employees given inducements from colluding 
> companies. My patch won't prevent this sort of behaviour but it does 
> remove a directive that would help facilitate it.
> 
> Greg, if you want to see more performance reviews, the maturity model 
> could compel companies to provide unsolicited feedback, instead of seek it 
> from an arbitrary source. Would you be amenable to a revised patch along 
> those lines?

It was never about *companies* providing unsolicited feedback, but
rather the upstream Linux kerenl development community:

Level 4
=======

* Organizations will consider community member feedback in official
  performance reviews.

Level 5
=======

* Organizations will actively seek out community member feedback as a
  factor in official performance reviews.

I could see making this be more explicit by spelling out "upstream
development community" and "regarding their upstream contributions".
But I'm not sure where you thought it was about "getting *from*
companies".

The reality is that many, if not most of the key Linux kernel
developer leaders are employed by companies.  And so we are quite well
practiced at being able to put on our "open source leader hat" --- and
explicitly telling people that this is what we are doing --- as well
as being able to explain when we are relying the requirements from a
particular company, usually when we are explianing what motiviated a
particular code contribution.

So if I were asked to give a recommendation for an employee working at
Company I, and I'm working at Company G, I'm perfectly capable of
saying, this is what this person has done as an upstream developer,
and based on her upstream contributions, you should definitely promote
her.  And this has actually happened, BTW.  If we could encourage more
companies to sek out or accept more feedback from community members,
that would certainly be a good thing.

Cheers,

						- Ted

Re: [PATCH] Documentation: Linux Contribution Maturity Model and the wider community

[Finn Thain]

On Tue, 20 Jun 2023, Theodore Ts'o wrote:

> On Tue, Jun 20, 2023 at 01:54:23PM +1000, Finn Thain wrote:
> > 
> > I suspect that counting commits may be the wrong metric (I can think 
> > of better ones).
> 
> As far as whether counting commits is the wrong metric, it doesn't 
> matter if you look at lines of code changes, attendance at conferences 
> such as the Linux Plumbers Conference, or participation on weekly 
> subsystem calls or video conferences, it's fair to say that that the 
> vast majority of those current developers work for companies.
> 

The metric I'd use would be one that considers the benefit to the wider 
community. The proportion of developers who are on a payroll is relevant 
to that -- inasmuchas users are customers.

In general, FLOSS users are not customers because if they could buy what 
they wanted, they would not have to build, modify, integrate or support 
it.

> > But if that's what we have, the lack of commits from non-profit 
> > organizations is a situation that might actually be improved by 
> > changes like the ones I'm advocating.
> 
> Not only have you not provided any evidence for your thesis (which to be 
> fair is hard since what you are positing is a hypothetical), but you 
> haven't even provided a theory why you believe that to be true!
> 

Well, evidence is good. The literature has evidence to support the 
effectiveness of other maturity models when put into practice in an open 
source setting.

Is there evidence to support the notion that your Linux Contributor 
Maturity Model (in its present, unpatched form) will bring the anticipated 
benefits, should companies choose to embrace it (and perhaps replace 
whatever methodologies they presently use)?

The technical projects under the purview of FINOS require a contributor 
license agreement. This has historically been a difficult pill for some 
contributors to swallow, so it's hard to imagine widespread adoption of 
the entire FINOS methodolgy (absent some kind of leverage).

AFAICT, non-profit organizations are not considered by the FINOS 
formulation of an open source maturity model, whereas, the other open 
source maturity models I came across when skimming the literature do 
indeed consider those organizations.

The FINOS OSMM is designed to "ensure maximum profitability" while 
minimizing risk. 
https://github.com/finos-labs/osmm/blob/main/docs/user/OSMM-Explanations.md

> I *do* have a theory for what we've observed, which is that developers 
> like to have food with their meals, and the amount of time and effort to 
> do serious kernel work is significant.  And so companies are the ones 
> who can fund the engineers which are spending a good proportion of their 
> time working on Linux kerenl development.  These is why we see a huge 
> amount of work from people who work for Linaro, Intel, Red Hat, Google, 
> IBM, Nvidia, Facebook, Oracle, etc. both in terms of code contributed to 
> the kernel, code reviews of other people's code submission, design 
> discussions on LKML and in various "hallway track conversations" at the 
> verious Linux conferences/workshops, etc.
> 
> As far as non-profit organizations are concerned, most of them have very 
> tightly defined mission.  That mission might be mostly unrelated to the 
> kernel, except as a customer of the kernel --- for example, the Mozilla 
> and Eclipse Foundation --- or it might be Linux adjacent, such as "make 
> a good distributions ala Debian, Arch, Fedora, etc."  The people at 
> these non-profits might be volunteers (this is mostly the case for 
> Debian), or they might be paid engineers based on the corporate 
> sponsoprs of the non-profits (for example) or engineers who are paid by 
> a company, but who are "on loan" to the non-profit organization (for 
> example, in the Apache Software Foundation this is a common pattern).  
> Either way, though, those non-profits tend to have a very tightly 
> focused mission, and it tends not to be one which requires a large 
> amount of kernel development.
> 
> You appear to have a very different model of how non-profits might 
> approach the Linux kernel --- could you go into more detail about why 
> they might want to contribute to the Linux kernel, and how we might 
> encourage them to contribute more engineering effort?
> 

Sure. Here's a recent example, in which a not-for-profit volunteer might 
have been granted an opportunity to work upstream: 
https://lore.kernel.org/all/129c9d5e-213a-80c9-092e-dc1dcf38ae3e@linux-m68k.org/

The driver in question may may not be commercially viable, but that 
doesn't matter, if the intention is to foster new maintainers and increase 
the talent pool. And the problem ostensibly being addressed in the Linux 
Contributor Maturity Model is a shortage of maintainers.

I don't have a magic bullet to solve that problem (which is not just a 
Linux problem) but I'll make a few observations.

- Maintainers should be "automating themselves out of a job" to whatever 
  extent this is possible.  git is a good example of this, as is all of 
  the tooling and workflow automation that grew out of that (e.g. gitlab).

  Because the Linux project is structured as a heirarchy, I think Linus 
  and senior maintainers have a crucial role here. I don't think it's a 
  co-incidence that git was the brainchild of the top maintainer.

  Making the maintainer role more lucrative will provide a disincentive 
  for more automation (with or without level 5 performance reviews) unless 
  remuneration is tied to metrics that reflect maintainer effectiveness.

- The roles of maintainer and reviewer should be taught in universities at 
  a post-graduate level to increase the talent pool.

- On the whole, I don't think remuneration or training can solve the 
  problem. I do think automation and tooling can do it.

  To develop that technology, subsystem maintainers must collaborate on 
  process, automation and tooling. That means they must remedy the 
  balkanization that presently exists across subsystems.

  I realize that experimentation and risk-taking are part of the reason 
  why Linux is the success that it is. However, at some point, senior 
  maintainers have to decide, for example, "the model used by subsystem A 
  is _measurably_ better than the process used by subsystem B, so the 
  former technique will become mandatory and then collaboratively improved 
  upon."

  So, we come back to metrics again. (As you would know, "you can't 
  improve what you can't measure.)

> > > I'm not sure how this document would "hinder collaboration" with 
> > > non-profit organizations and individuals.  Could you say more about 
> > > your concern regarding how this undesireable outcome would happen in 
> > > practice?
> > 
> > I believe that I've now addressed this in my message to Greg.
> 
> Well, no, you haven't.  More below....
> 
> On Tue, Jun 20, 2023 at 01:48:59PM +1000, Finn Thain wrote: (in reply to 
> Greg)
> > 
> > Bonuses and salaries are tied to performance reviews so the hazard 
> > here are clear. Level 5 compels companies to seek feedback and 
> > naturally they will seek it from companies who share their goals. You 
> > ask too much of employees if you expect them to put aside the 
> > corporate agendas and pursue the interests of the wider community.
> 
> I was a hobbyist from 1991 to 1999 (I was the first North American linux 
> kernel developer, and at the time my day job was tech lead for the MIT 
> KerBeros Team and I also served on the IETF Security Area Directorate 
> and was one of the IPSEC working group chairs), and then from 1999 until 
> present, I've worked for companies (first VA Linux Systems, then the IBM 
> Linux Technology Cetner, and now at Google).  So I think I know 
> something about how employees balance the needs of the Linux Kernel 
> community and the needs of their employer.
> 

I don't mean to lecture you, Ted. I have great admiration for your 
considerable contributions and insight. Also, I benefit from extfs every 
day and for that I'm grateful.

> > Countless lawsuits over the last few decades made it abundantly clear 
> > that the goals of companies often diverge from those of the wider 
> > FLOSS community.
> > 
> > Consider all of the open source code thrown over the wall, the binary 
> > blobs, the binary modules, the built-in obsolescence, the devices 
> > shipped with vulnerabilities now reduced to e-waste because they 
> > cannot be fixed, the vendor lock-in strategies, the walled gardens, 
> > the surveillance etc.
> 
> There haven't been *that* many lawsuits, and while there have been some 
> bad actors, there have also been many, MANY examples of companies that 
> have contributed in highly positive ways.  For example, well over a 
> decade ago, IBM started requiring that their peripheral card suppliers 
> (e.g., network cards, SCSI host bus adapters, etc.) that it would be a 
> requirement that thosse companies providing those peripherals MUST have 
> upstream Linux device drivers as a condition of the procurement 
> contract.
> 
> And the more enlightened companies *do* understood that out-of-tree 
> patches are technical debt, and to get drivers, patches, etc., upstream 
> in the long run would save them huge amounts of effort.  So there are 
> plenty of ways in which the meeting the goals of the FLOSS comunity is 
> ultimately, good towards achieving the goals of for-profit companies.
> 

Indeed.

> > To my jaded mind, it is obvious that such reprehensible strategies can 
> > be advanced by co-operative employees given inducements from colluding 
> > companies. My patch won't prevent this sort of behaviour but it does 
> > remove a directive that would help facilitate it.
> > 
> > Greg, if you want to see more performance reviews, the maturity model 
> > could compel companies to provide unsolicited feedback, instead of 
> > seek it from an arbitrary source. Would you be amenable to a revised 
> > patch along those lines?
> 
> It was never about *companies* providing unsolicited feedback, but 
> rather the upstream Linux kerenl development community:
> 
> Level 4
> =======
> 
> * Organizations will consider community member feedback in official 
>   performance reviews.
> 
> Level 5
> =======
> 
> * Organizations will actively seek out community member feedback as a 
>   factor in official performance reviews.
> 
> I could see making this be more explicit by spelling out "upstream 
> development community" and "regarding their upstream contributions". But 
> I'm not sure where you thought it was about "getting *from* companies".
> 

I know that was not the intention, but I think that the incentives work 
against the intention, and it need not be so.

> The reality is that many, if not most of the key Linux kernel developer 
> leaders are employed by companies.  And so we are quite well practiced 
> at being able to put on our "open source leader hat" --- and explicitly 
> telling people that this is what we are doing --- as well as being able 
> to explain when we are relying the requirements from a particular 
> company, usually when we are explianing what motiviated a particular 
> code contribution.
> 
> So if I were asked to give a recommendation for an employee working at 
> Company I, and I'm working at Company G, I'm perfectly capable of 
> saying, this is what this person has done as an upstream developer, and 
> based on her upstream contributions, you should definitely promote her.  
> And this has actually happened, BTW.  If we could encourage more 
> companies to sek out or accept more feedback from community members, 
> that would certainly be a good thing.
> 
> Cheers,
> 
> 						- Ted
> 
> 

Re: [PATCH] Documentation: Linux Contribution Maturity Model and the wider community

[Greg Kroah-Hartman]

On Wed, Jun 21, 2023 at 11:51:19AM +1000, Finn Thain wrote:
> > You appear to have a very different model of how non-profits might 
> > approach the Linux kernel --- could you go into more detail about why 
> > they might want to contribute to the Linux kernel, and how we might 
> > encourage them to contribute more engineering effort?
> > 
> 
> Sure. Here's a recent example, in which a not-for-profit volunteer might 
> have been granted an opportunity to work upstream: 
> https://lore.kernel.org/all/129c9d5e-213a-80c9-092e-dc1dcf38ae3e@linux-m68k.org/
> 
> The driver in question may may not be commercially viable, but that 
> doesn't matter, if the intention is to foster new maintainers and increase 
> the talent pool. And the problem ostensibly being addressed in the Linux 
> Contributor Maturity Model is a shortage of maintainers.

I would NEVER recommend ANYONE picking up obsolete hardware and trying
to get it to work to maintain the driver if NO ONE is actually using the
stuff.  That should not be for a not-for-profit to maintain as
obviously, no one uses it.

It's up to those that need/use the code to help maintain it, don't ask
not-for-profit groups to maintain and support code that no one uses,
that's a sure way to waste resources all around.

So that's a good example of how our ecosystem works properly, if no one
needs the code, it gets dropped.  Don't ask for it to come back without
real users who are invested in it please.

thanks,

greg k-h

Re: [PATCH] Documentation: Linux Contribution Maturity Model and the wider community

[Finn Thain]

On Wed, 21 Jun 2023, Greg Kroah-Hartman wrote:

> On Wed, Jun 21, 2023 at 11:51:19AM +1000, Finn Thain wrote:
> > > You appear to have a very different model of how non-profits might 
> > > approach the Linux kernel --- could you go into more detail about 
> > > why they might want to contribute to the Linux kernel, and how we 
> > > might encourage them to contribute more engineering effort?
> > > 
> > 
> > Sure. Here's a recent example, in which a not-for-profit volunteer 
> > might have been granted an opportunity to work upstream: 
> > https://lore.kernel.org/all/129c9d5e-213a-80c9-092e-dc1dcf38ae3e@linux-m68k.org/
> > 
> > The driver in question may may not be commercially viable, but that 
> > doesn't matter, if the intention is to foster new maintainers and 
> > increase the talent pool. And the problem ostensibly being addressed 
> > in the Linux Contributor Maturity Model is a shortage of maintainers.
> 
> I would NEVER recommend ANYONE picking up obsolete hardware and trying 
> to get it to work to maintain the driver if NO ONE is actually using the 
> stuff.  That should not be for a not-for-profit to maintain as 
> obviously, no one uses it.
> 
> It's up to those that need/use the code to help maintain it, don't ask 
> not-for-profit groups to maintain and support code that no one uses, 
> that's a sure way to waste resources all around.
> 

Actually, that patch was offered without any prompting from me. But you're 
quite right -- I would have prompted it, had I had the oppportunity.

> So that's a good example of how our ecosystem works properly, if no one 
> needs the code, it gets dropped.  Don't ask for it to come back without 
> real users who are invested in it please.
> 

You mentioned wasted resources. If you want to generate e-waste, remove 
drivers from the kernel. If you want to prevent e-waste, add drivers for 
obsolete hardware and then watch the user count climb from zero as devices 
get pulled from drawers and dusted off.

Anyway, your reaction is an interesting example of strong feelings in the 
community as to how contributed code should or should not be used. E.g. 
some get upset if their code runs on weapons systems, others get upset if 
the latest release might not run on suitable hardware in the immediate 
future. Some add or remove licence terms according to their convictions.

If there was consensus, it might be feasible to give a formula for 
"recognized usage" which could be quantified. From there we could create a 
kind of heat map to show which commits, maintainers, processes, models, 
modules etc. were the most "useful" within some time interval.

Re: [PATCH] Documentation: Linux Contribution Maturity Model and the wider community

[Greg Kroah-Hartman]

On Thu, Jun 22, 2023 at 05:02:10PM +1000, Finn Thain wrote:
> On Wed, 21 Jun 2023, Greg Kroah-Hartman wrote:
> 
> > On Wed, Jun 21, 2023 at 11:51:19AM +1000, Finn Thain wrote:
> > > > You appear to have a very different model of how non-profits might 
> > > > approach the Linux kernel --- could you go into more detail about 
> > > > why they might want to contribute to the Linux kernel, and how we 
> > > > might encourage them to contribute more engineering effort?
> > > > 
> > > 
> > > Sure. Here's a recent example, in which a not-for-profit volunteer 
> > > might have been granted an opportunity to work upstream: 
> > > https://lore.kernel.org/all/129c9d5e-213a-80c9-092e-dc1dcf38ae3e@linux-m68k.org/
> > > 
> > > The driver in question may may not be commercially viable, but that 
> > > doesn't matter, if the intention is to foster new maintainers and 
> > > increase the talent pool. And the problem ostensibly being addressed 
> > > in the Linux Contributor Maturity Model is a shortage of maintainers.
> > 
> > I would NEVER recommend ANYONE picking up obsolete hardware and trying 
> > to get it to work to maintain the driver if NO ONE is actually using the 
> > stuff.  That should not be for a not-for-profit to maintain as 
> > obviously, no one uses it.
> > 
> > It's up to those that need/use the code to help maintain it, don't ask 
> > not-for-profit groups to maintain and support code that no one uses, 
> > that's a sure way to waste resources all around.
> > 
> 
> Actually, that patch was offered without any prompting from me. But you're 
> quite right -- I would have prompted it, had I had the oppportunity.
> 
> > So that's a good example of how our ecosystem works properly, if no one 
> > needs the code, it gets dropped.  Don't ask for it to come back without 
> > real users who are invested in it please.
> > 
> 
> You mentioned wasted resources. If you want to generate e-waste, remove 
> drivers from the kernel. If you want to prevent e-waste, add drivers for 
> obsolete hardware and then watch the user count climb from zero as devices 
> get pulled from drawers and dusted off.

Based on our experience, opld devices are not being pulled from
anywhere, and if they were being used, they almost always take more
power than existing devices, so it's hard to justify the use of them at
times given energy prices in many parts of the world.

> Anyway, your reaction is an interesting example of strong feelings in the 
> community as to how contributed code should or should not be used.

I did not pass any judgement on how the code should or should not be
used.  All I am saying is that this code is not being used at all.

> If there was consensus, it might be feasible to give a formula for 
> "recognized usage" which could be quantified. From there we could create a 
> kind of heat map to show which commits, maintainers, processes, models, 
> modules etc. were the most "useful" within some time interval.

Determining code use is difficult given that we are not going to add
tracking to the kernel source for obvious reasons, so this is a
non-starter, as you know.

greg k-h

Re: [PATCH] Documentation: Linux Contribution Maturity Model and the wider community

[Finn Thain]

On Thu, 22 Jun 2023, Greg Kroah-Hartman wrote:

> 
> > If there was consensus, it might be feasible to give a formula for 
> > "recognized usage" which could be quantified. From there we could 
> > create a kind of heat map to show which commits, maintainers, 
> > processes, models, modules etc. were the most "useful" within some 
> > time interval.
> 
> Determining code use is difficult given that we are not going to add 
> tracking to the kernel source for obvious reasons, so this is a 
> non-starter, as you know.
> 

It never crossed my mind that the kernel might "phone home". Counting 
execution events (privately) doesn't work either because it seems 
hopelessly skewed towards fast processors, clusters and fast paths.
Or am I missing something?

Re: [PATCH] Documentation: Linux Contribution Maturity Model and the wider community

[Theodore Ts'o]

On Thu, Jun 22, 2023 at 05:02:10PM +1000, Finn Thain wrote:
> 
> You mentioned wasted resources. If you want to generate e-waste, remove 
> drivers from the kernel. If you want to prevent e-waste, add drivers for 
> obsolete hardware and then watch the user count climb from zero as devices 
> get pulled from drawers and dusted off.

You seem to making a lot of assumptions here, with no evidence to back
up your assertions.  The driver question is from twenty years ago, and
talks to SCSI cards using LVD (low-voltage diferential) SCSI 3 cables
(for 160 MB/s).  SCSI Disks from that era are typically 20GB to 30GB.
Compare that to modern SATA disks today that are 10-18 TB in size, and
SATA-3 transfers data at 600 MB/s.

So you are assuming (a) that people just "happen" to have ancient, 20
year old cards justlying around in drawers, (b) they have 20 year old
SCSI disks and SCSI cables that these cards would actually talk to,
and (c) they would think it would make sense from a power, space, and
cooling perspective to take this kind of antique storage solutions are
use it for actually storing data.

> Anyway, your reaction is an interesting example of strong feelings in the 
> community as to how contributed code should or should not be used. E.g. 
> some get upset if their code runs on weapons systems, others get upset if 
> the latest release might not run on suitable hardware in the immediate 
> future. Some add or remove licence terms according to their convictions.

Um, I don't even know where this came from.  In any case, the Linux
Kernel is licensed under the GPL2, which, like all Open Source
compliant licenses, does not distribute against fields of endeavor
(such as weapons systems, etc.)

As far as getting upset if the latest release doesn't run on "suitable
hardware", if they are upset they can submit a bug report, or better
yet, submit a patch to address the situation.  What people seem to
forget is that free software does not mean that people will fix your
software for your use case for free.  It means that *you* are free to
fix the software, or to pay someone to fix the software in question.

> If there was consensus, it might be feasible to give a formula for 
> "recognized usage" which could be quantified. From there we could create a 
> kind of heat map to show which commits, maintainers, processes, models, 
> modules etc. were the most "useful" within some time interval.

The best we have is we can see what users submit bug reports about
(especially, for example, when we discover that some driver was
accidentally broken a few years ago due to the need to upgrade code to
use newer API's as we improve and refactor code, and no one has
complained about the driver being used --- that's a good hint that no
one cares), and what individuals and companies choose to spend time
working to improve certain parts of the kernel.  If code is under
active maintenance, then it's worth *someone's* time to keep
maintained.

And of course, if remove a driver because it is unmaintained and is
for obsolete hardware, if someone shows up saying (a) they care about
that driver, and (b) they are willing to volunteer to maintain the
driver, or are willing to pay someone to maintain the driver, and they
have contracted with XYZ developer working for ABC company, then it's
super simple to revert the driver removal.  It is, after all, only a
"git revert" away.

I do have to concur with Greg that relying on this as way to get new
people to be work on Linux kernel is a *terrible* idea.  The number of
people who are interested in retro-computing is quite small, in my
experience.

Cheers,

					- Ted

Re: [PATCH] Documentation: Linux Contribution Maturity Model and the wider community

[Finn Thain]

On Thu, 22 Jun 2023, Theodore Ts'o wrote:

> On Thu, Jun 22, 2023 at 05:02:10PM +1000, Finn Thain wrote:
> > 
> > You mentioned wasted resources. If you want to generate e-waste, 
> > remove drivers from the kernel. If you want to prevent e-waste, add 
> > drivers for obsolete hardware and then watch the user count climb from 
> > zero as devices get pulled from drawers and dusted off.
> 
> You seem to making a lot of assumptions here, with no evidence to back 
> up your assertions.  The driver question is from twenty years ago, and 
> talks to SCSI cards using LVD (low-voltage diferential) SCSI 3 cables 
> (for 160 MB/s).  SCSI Disks from that era are typically 20GB to 30GB. 
> Compare that to modern SATA disks today that are 10-18 TB in size, and 
> SATA-3 transfers data at 600 MB/s.
> 
> So you are assuming (a) that people just "happen" to have ancient, 20 
> year old cards justlying around in drawers, (b) they have 20 year old 
> SCSI disks and SCSI cables that these cards would actually talk to, and 
> (c) they would think it would make sense from a power, space, and 
> cooling perspective to take this kind of antique storage solutions are 
> use it for actually storing data.
> 

No, those are not my assumptions, those are my observations.

Also, you've incorrectly conflated my comment about "devices in drawers" 
(which was a reference to old Android phones and the like) with an old 
thread about an HBA driver. The former comment goes to the value of old 
code. The latter goes to the size of the talent pool.

> > Anyway, your reaction is an interesting example of strong feelings in 
> > the community as to how contributed code should or should not be used. 
> > E.g. some get upset if their code runs on weapons systems, others get 
> > upset if the latest release might not run on suitable hardware in the 
> > immediate future. Some add or remove licence terms according to their 
> > convictions.
> 
> Um, I don't even know where this came from.

I'll explain it.

> In any case, the Linux Kernel is licensed under the GPL2, which, like 
> all Open Source compliant licenses, does not distribute against fields 
> of endeavor (such as weapons systems, etc.)
> 

As I understand it, supporting ancient hardware in certain sectors is 
highly profitable, where red tape prevents those customers from upgrading.

> As far as getting upset if the latest release doesn't run on "suitable 
> hardware", if they are upset they can submit a bug report, or better 
> yet, submit a patch to address the situation. 

I think you've missed my point, which was that some maintainers require 
that released code is executed promptly otherwise it should be deleted. 
Please see also, 
https://lore.kernel.org/all/7c2a6687-9c4e-efed-5e25-774b582e9a27@linux-m68k.org/

> What people seem to forget is that free software does not mean that 
> people will fix your software for your use case for free.  It means that 
> *you* are free to fix the software, or to pay someone to fix the 
> software in question.
> 
> > If there was consensus, it might be feasible to give a formula for 
> > "recognized usage" which could be quantified. From there we could 
> > create a kind of heat map to show which commits, maintainers, 
> > processes, models, modules etc. were the most "useful" within some 
> > time interval.
> 
> The best we have is we can see what users submit bug reports about 
> (especially, for example, when we discover that some driver was 
> accidentally broken a few years ago due to the need to upgrade code to 
> use newer API's as we improve and refactor code, and no one has 
> complained about the driver being used --- that's a good hint that no 
> one cares), and what individuals and companies choose to spend time 
> working to improve certain parts of the kernel.  If code is under active 
> maintenance, then it's worth *someone's* time to keep maintained.
> 
> And of course, if remove a driver because it is unmaintained and is for 
> obsolete hardware, if someone shows up saying (a) they care about that 
> driver, and (b) they are willing to volunteer to maintain the driver, or 
> are willing to pay someone to maintain the driver, and they have 
> contracted with XYZ developer working for ABC company, then it's super 
> simple to revert the driver removal.  It is, after all, only a "git 
> revert" away.
> 
> I do have to concur with Greg that relying on this as way to get new 
> people to be work on Linux kernel is a *terrible* idea.  The number of 
> people who are interested in retro-computing is quite small, in my 
> experience.
> 

Given that products like mobile phones etc. often get made obsolete within 
a few years from launch, my guess is that billions of users are now 
interested in retro-computing.

> Cheers,
> 
> 					- Ted
> 

Re: [Tech-board-discuss] [PATCH] Documentation: Linux Contribution Maturity Model and the wider community

[Mark Brown]

[-- Attachment #1: Type: text/plain, Size: 2078 bytes --]

On Fri, Jun 23, 2023 at 10:52:08AM +1000, Finn Thain wrote:
> On Thu, 22 Jun 2023, Theodore Ts'o wrote:

> > As far as getting upset if the latest release doesn't run on "suitable 
> > hardware", if they are upset they can submit a bug report, or better 
> > yet, submit a patch to address the situation. 

> I think you've missed my point, which was that some maintainers require 
> that released code is executed promptly otherwise it should be deleted. 
> Please see also, 
> https://lore.kernel.org/all/7c2a6687-9c4e-efed-5e25-774b582e9a27@linux-m68k.org/

Looking at that thread I'm not sure that's an entirely accurate reading
of what was being suggested.

> > And of course, if remove a driver because it is unmaintained and is for 
> > obsolete hardware, if someone shows up saying (a) they care about that 
> > driver, and (b) they are willing to volunteer to maintain the driver, or 
> > are willing to pay someone to maintain the driver, and they have 
> > contracted with XYZ developer working for ABC company, then it's super 
> > simple to revert the driver removal.  It is, after all, only a "git 
> > revert" away.

> > I do have to concur with Greg that relying on this as way to get new 
> > people to be work on Linux kernel is a *terrible* idea.  The number of 
> > people who are interested in retro-computing is quite small, in my 
> > experience.

> Given that products like mobile phones etc. often get made obsolete within 
> a few years from launch, my guess is that billions of users are now 
> interested in retro-computing.

The whole situation with embedded devices and their lifespans is a bit
different to that for a lot of more PC style hardware, and TBH if anyone
is actually interested in the hardware it's much more likely that the
support will be actively maintained and the issue just won't arise.  The
pushback with obsolete devices is more about a lack of anyone paying
attention to them and them causing trouble as a result of that than it
is about cases where someone is actively looking at them.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [Tech-board-discuss] [PATCH] Documentation: Linux Contribution Maturity Model and the wider community

[Steven Rostedt]

On Wed, 21 Jun 2023 11:51:19 +1000 (AEST)
Finn Thain <fthain@linux-m68k.org> wrote:

> - Maintainers should be "automating themselves out of a job" to whatever 
>   extent this is possible.  git is a good example of this, as is all of 
>   the tooling and workflow automation that grew out of that (e.g. gitlab).

I agree with the above statement.

> 
>   Because the Linux project is structured as a heirarchy, I think Linus 
>   and senior maintainers have a crucial role here. I don't think it's a 
>   co-incidence that git was the brainchild of the top maintainer.

True.

> 
>   Making the maintainer role more lucrative will provide a disincentive 
>   for more automation (with or without level 5 performance reviews) unless 
>   remuneration is tied to metrics that reflect maintainer effectiveness.

I'm not sure I totally understand your point above. I do not think that
making the maintainer role more lucrative provides a disincentive for more
automation. I'm constantly trying to add more automation to my process.
That's why I created ktest.pl, and constantly fiddling with patchwork to
get patch state automatically updated when things move from different
branches and git trees.

If your point is mainly the second part of that paragraph, which is to tie
in metrics to reflect maintainer effectiveness, then I think I agree with
you there. One metric is simply the time a patch is ignored by a
maintainer on a mailing list (where the maintainer is Cc'd and it is
obvious the patch belongs to their subsystem). I know I fail at that,
especially when my work is pushing me to focus on other things.

-- Steve

Re: [Tech-board-discuss] [PATCH] Documentation: Linux Contribution Maturity Model and the wider community

[Finn Thain]

On Wed, 21 Jun 2023, Steven Rostedt wrote:

> 
> > 
> >   Making the maintainer role more lucrative will provide a 
> >   disincentive for more automation (with or without level 5 
> >   performance reviews) unless remuneration is tied to metrics that 
> >   reflect maintainer effectiveness.
> 
> I'm not sure I totally understand your point above. I do not think that 
> making the maintainer role more lucrative provides a disincentive for 
> more automation. 

You're right -- it's a moot point (whether paying people more will reward 
underperformers) since it all depends on the performance metric. I was 
assuming a metric that reflects my own bias.

Measurement, was Re: [Tech-board-discuss] [PATCH] Documentation: Linux Contribution Maturity Model and the wider community

[Finn Thain]

On Wed, 21 Jun 2023, Steven Rostedt wrote:

> 
> If your point is mainly the second part of that paragraph, which is to 
> tie in metrics to reflect maintainer effectiveness, then I think I agree 
> with you there. One metric is simply the time a patch is ignored by a 
> maintainer on a mailing list (where the maintainer is Cc'd and it is 
> obvious the patch belongs to their subsystem). I know I fail at that, 
> especially when my work is pushing me to focus on other things.
> 

A useful metric when pushing for a higher patch rate is the rework rate.

I have found that 'Fixes' tags can be used to quantify this. I don't have 
scripts to do so but others probably do. (My purpose at the time was to 
quantify my own rework rate by counting my own commit hashes when they 
appeared in subsequent 'Fixes' tags.) Note that a low 'Fixes' count could 
indicate inadequate bug reporting processes so additional metrics may be 
needed.

Where the practices relating to 'Fixes' tagging and bug reporting are 
uniform across subsystems, it might be possible to compare the diverse 
processes and methodologies presently in use.

BTW. I assume that 'Fixes' tags are already being used to train AI models 
to locate bugs in existing code. If this could be used to evaluate new 
patches when posted, it might make the code review process more efficient.

The same approach could probably be generalized somewhat. For example, a 
'Modernizes' tag might be used to train an AI model to target design 
patterns that are being actively replaced anywhere in the code base.

The real pay-off from this kind of automation is that an improvement made 
by any reviewer gets amplified so as to reach across many subsystems and 
mailing lists -- but only when the automation gets scaled up and widely 
deployed. We already see this effect with Coccinelle semantic patches.

Re: [Tech-board-discuss] Measurement, was Re: [PATCH] Documentation: Linux Contribution Maturity Model and the wider community

[Greg Kroah-Hartman]

On Sat, Jul 01, 2023 at 11:46:18AM +1000, Finn Thain wrote:
> BTW. I assume that 'Fixes' tags are already being used to train AI models 
> to locate bugs in existing code. If this could be used to evaluate new 
> patches when posted, it might make the code review process more efficient.

That has been happening for many many years now with papers being
published about it and many conference presentations.  It shouldn't be a
secret it's been happening and directly helping with stable kernel
maintenance for a long time.

thanks,

greg k-h

Re: [Tech-board-discuss] Measurement, was Re: [PATCH] Documentation: Linux Contribution Maturity Model and the wider community

[Finn Thain]

On Sat, 1 Jul 2023, Greg Kroah-Hartman wrote:

> On Sat, Jul 01, 2023 at 11:46:18AM +1000, Finn Thain wrote:
> > BTW. I assume that 'Fixes' tags are already being used to train AI 
> > models to locate bugs in existing code. If this could be used to 
> > evaluate new patches when posted, it might make the code review 
> > process more efficient.
> 
> That has been happening for many many years now with papers being 
> published about it and many conference presentations.  It shouldn't be a 
> secret it's been happening and directly helping with stable kernel 
> maintenance for a long time.
> 

Many years ago it struck me that the deficiencies of checkpatch.pl could 
be addressed with coccinelle but I still don't see this happening on the 
lists I read.

I see reviewers being spared from having to examine many flawed patches 
because the zero-day bot intercepted them and fed them to static 
analyzers. But I still don't see coccinelle being used to the same end 
i.e. to reduce the burden on reviewers and maintainers.

Has no-one tried it, or did it not work out?

Re: [PATCH] Documentation: Linux Contribution Maturity Model and the wider community

[Finn Thain]

On Wed, 21 Jun 2023, Finn Thain wrote:

> The technical projects under the purview of FINOS require a contributor 
> license agreement. This has historically been a difficult pill for some 
> contributors to swallow, so it's hard to imagine widespread adoption of 
> the entire FINOS methodolgy

I'm afraid I may have mixed up CLA and copyright assignment. I can't seem 
to find the relevant CLA text. Sorry if I've mislead anyone.

Re: [PATCH] Documentation: Linux Contribution Maturity Model and the wider community

[Matthew Wilcox]

On Wed, Jun 21, 2023 at 11:51:19AM +1000, Finn Thain wrote:
> - The roles of maintainer and reviewer should be taught in universities at 
>   a post-graduate level to increase the talent pool.

Umm.  I can't say that I know anyone who studied computer science at a
post-graduate level and then became a Linux maintainer.  They probably
exist, but I'm not aware of them.  In contrast, I can name two people
who started a PhD in another subject and then got lured into Linux
development, abandoning their PhD.  I suspect most have a bachelors.
Some do not have a degree.

I don't think it's the role of a computer science department to do
this kind of teaching.  It feels more practical.  Now maybe it's part
of a software engineering curriculum, but then I don't think it's a
post-graduate topic.

It could also be something a professional society pushes.  The British
Computer Society were really into that kind of thing twenty-five years
ago when they were trying to persuade me to join.

Re: [PATCH] Documentation: Linux Contribution Maturity Model and the wider community

[Kees Cook]

On Mon, Jun 19, 2023 at 07:41:57PM +1000, Finn Thain wrote:
> The Linux Contribution Maturity Model methodology is notionally based on
> the Open source Maturity Model (OMM) which was in turn based on the
> Capability Maturity Model Integration (CMMI).
> 
> According to Petrinja et al., the goal of the OMM was to extend the CMMI
> so as to be useful both for companies and for communities [1][2]. However,
> the Linux Contribution Maturity Model considers only companies and
> businesses.
> 
> This patch addresses this bias as it could hinder collaboration with
> not-for-profit organisations and individuals, which would be a loss to
> any stakeholder.
> 
> Level 5 is amended to remove the invitation to exercise the same bias
> i.e. employees rewarded indirectly by other companies.
> 
> [1] Petrinja, E., Nambakam, R., Sillitti, A.: Introducing the
> OpenSource Maturity Model. In: 2nd Emerging Trends in FLOSS Research
> and Development Workshop at ICSE 2009, Vancouver, BC, Canada (2009)
> 
> [2] Wittmann, M., Nambakam, R.: Qualipso Deliverable A6.D1.6.3
> CMM-like model for OSS.
> 
> Cc: Theodore Ts'o <tytso@mit.edu>
> Cc: Kees Cook <keescook@chromium.org>
> Cc: Dan Williams <dan.j.williams@intel.com>
> Signed-off-by: Finn Thain <fthain@linux-m68k.org>
> ---
>  Documentation/process/contribution-maturity-model.rst | 9 ++++-----
>  1 file changed, 4 insertions(+), 5 deletions(-)
> 
> diff --git a/Documentation/process/contribution-maturity-model.rst b/Documentation/process/contribution-maturity-model.rst
> index b87ab34de22c..863a2e4c22e2 100644
> --- a/Documentation/process/contribution-maturity-model.rst
> +++ b/Documentation/process/contribution-maturity-model.rst
> @@ -62,8 +62,8 @@ Level 3
>  =======
>  
>  * Software Engineers are expected to review patches (including patches
> -  authored by engineers from other companies) as part of their job
> -  responsibilities
> +  authored by contributors from outside of the organization) as part of
> +  their job responsibilities

This seems fine to me.

>  * Contributing presentations or papers to Linux-related or academic
>    conferences (such those organized by the Linux Foundation, Usenix,
>    ACM, etc.), are considered part of an engineer’s work.
> @@ -103,7 +103,6 @@ Level 5
>  
>  * Upstream kernel development is considered a formal job position, with
>    at least a third of the engineer’s time spent doing Upstream Work.
> -* Organizations will actively seek out community member feedback as a
> -  factor in official performance reviews.

This really cannot be dropped -- companies must factor upstream work
into performance reviews or it will continue to be seen as "free time"
work, and employees won't be recognized for their upstream contributions.
If an org has no perf reviews, this item is already nullified, IMO.

>  * Organizations will regularly report internally on the ratio of
> -  Upstream Work to work focused on directly pursuing business goals.
> +  Upstream Work to work focused on directly pursuing the organisation's
> +  other goals.

This seems fine to me.

-Kees

-- 
Kees Cook

Re: [PATCH] Documentation: Linux Contribution Maturity Model and the wider community

[Finn Thain]

[-- Attachment #1: Type: text/plain, Size: 730 bytes --]

On Mon, 19 Jun 2023, Kees Cook wrote:

> > @@ -103,7 +103,6 @@ Level 5
> >  
> >  * Upstream kernel development is considered a formal job position, with
> >    at least a third of the engineer’s time spent doing Upstream Work.
> > -* Organizations will actively seek out community member feedback as a
> > -  factor in official performance reviews.
> 
> This really cannot be dropped -- companies must factor upstream work 
> into performance reviews or it will continue to be seen as "free time" 
> work, and employees won't be recognized for their upstream 
> contributions. If an org has no perf reviews, this item is already 
> nullified, IMO.
> 

I believe that I've now addressed this in my reply to Greg.