From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3C3EFEB64DA for ; Sat, 1 Jul 2023 01:45:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229694AbjGABpb (ORCPT ); Fri, 30 Jun 2023 21:45:31 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35192 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229447AbjGABp2 (ORCPT ); Fri, 30 Jun 2023 21:45:28 -0400 Received: from out3-smtp.messagingengine.com (out3-smtp.messagingengine.com [66.111.4.27]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A3A414201; Fri, 30 Jun 2023 18:45:25 -0700 (PDT) Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailout.nyi.internal (Postfix) with ESMTP id 7E6E45C0175; Fri, 30 Jun 2023 21:45:22 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute3.internal (MEProxy); Fri, 30 Jun 2023 21:45:22 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm2; t=1688175922; x=1688262322; bh=0032OLr1UDIHP BMzcHue27WJgCny0p4WJDgbUYNu8EA=; b=dJey4+Zm9gZf8ekEbrM21L90x3SOo pfIiLfBqJvJmY2Yw99ZFQEEqRCodmowN41Ml5cHDf43SMJ8JEzrP7Z4jgWEgRNAl DPkN6U9UUnnpxIGDuTeafOrVOo1GOplOon+nuOX9YAgPeVmpozOP2bxPN5hSZWmr kFYHKl1pDuDAiRvmmBumHESN5CsOzWkBAPXykV4U4KtJVXUQ+F6Zs6V36oiyrzXC ZmK9DQgYeEE2FRgMN6xQpYQcQaQnRWzF2jotz3sLhd21hamoZgMN4/CSwSZQDiLf xCrKPaBwaXS7CCvP+hIilKoT2jak54LF9vn+W4fUx/Zw6gUGZa/Mlz8WQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedviedrtdejgdeglecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpeffhffvvefujgfkfhggtgesthdtredttddtvdenucfhrhhomhephfhinhhnucfv hhgrihhnuceofhhthhgrihhnsehlihhnuhigqdhmieekkhdrohhrgheqnecuggftrfgrth htvghrnhepleeuheelheekgfeuvedtveetjeekhfffkeeffffftdfgjeevkeegfedvueeh ueelnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepfh hthhgrihhnsehlihhnuhigqdhmieekkhdrohhrgh X-ME-Proxy: Feedback-ID: i58a146ae:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Fri, 30 Jun 2023 21:45:19 -0400 (EDT) Date: Sat, 1 Jul 2023 11:46:18 +1000 (AEST) From: Finn Thain To: Steven Rostedt cc: Theodore Ts'o , linux-doc@vger.kernel.org, tech-board-discuss@lists.linux-foundation.org, Greg Kroah-Hartman , linux-kernel@vger.kernel.org Subject: Measurement, was Re: [Tech-board-discuss] [PATCH] Documentation: Linux Contribution Maturity Model and the wider community In-Reply-To: <20230621100845.12588f48@gandalf.local.home> Message-ID: <1f5b0227-dbf6-4294-8532-525b3e405dc2@linux-m68k.org> References: <20230620212502.GI286961@mit.edu> <5490402b-8b9f-f52d-3896-41090e639e51@linux-m68k.org> <20230621100845.12588f48@gandalf.local.home> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, 21 Jun 2023, Steven Rostedt wrote: > > If your point is mainly the second part of that paragraph, which is to > tie in metrics to reflect maintainer effectiveness, then I think I agree > with you there. One metric is simply the time a patch is ignored by a > maintainer on a mailing list (where the maintainer is Cc'd and it is > obvious the patch belongs to their subsystem). I know I fail at that, > especially when my work is pushing me to focus on other things. > A useful metric when pushing for a higher patch rate is the rework rate. I have found that 'Fixes' tags can be used to quantify this. I don't have scripts to do so but others probably do. (My purpose at the time was to quantify my own rework rate by counting my own commit hashes when they appeared in subsequent 'Fixes' tags.) Note that a low 'Fixes' count could indicate inadequate bug reporting processes so additional metrics may be needed. Where the practices relating to 'Fixes' tagging and bug reporting are uniform across subsystems, it might be possible to compare the diverse processes and methodologies presently in use. BTW. I assume that 'Fixes' tags are already being used to train AI models to locate bugs in existing code. If this could be used to evaluate new patches when posted, it might make the code review process more efficient. The same approach could probably be generalized somewhat. For example, a 'Modernizes' tag might be used to train an AI model to target design patterns that are being actively replaced anywhere in the code base. The real pay-off from this kind of automation is that an improvement made by any reviewer gets amplified so as to reach across many subsystems and mailing lists -- but only when the automation gets scaled up and widely deployed. We already see this effect with Coccinelle semantic patches.