From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.4 required=3.0 tests=DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CB71DFC6199 for ; Fri, 8 Nov 2019 21:20:20 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8494A207FA for ; Fri, 8 Nov 2019 21:20:20 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="DEvIO64O" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8494A207FA Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:60598 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iTBg3-0000xO-JT for qemu-devel@archiver.kernel.org; Fri, 08 Nov 2019 16:20:19 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:57079) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iTBeO-000846-9p for qemu-devel@nongnu.org; Fri, 08 Nov 2019 16:18:38 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iTBeK-0001d6-Lt for qemu-devel@nongnu.org; Fri, 08 Nov 2019 16:18:36 -0500 Received: from mail-wr1-x443.google.com ([2a00:1450:4864:20::443]:44758) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1iTBeK-0001bJ-Dq; Fri, 08 Nov 2019 16:18:32 -0500 Received: by mail-wr1-x443.google.com with SMTP id f2so8519055wrs.11; Fri, 08 Nov 2019 13:18:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=58kR9q2cEK71h1k3gjJqWvSSjyB6wHO/TzcyeXj29EE=; b=DEvIO64O3J0d45xVtJxaP4AsifeRtUKsE+B0WAIMxQxDmQY7e4KpX0Dz/tIv9jDYXz Gmxxof3h+Wvet5r52Np70JC5KBsb1M0PeNCunB3JyM0HVBJzWYswNu7VgvgFmszePLGE zMf0jS/oU119E5FnmqzmT4tGjl00q5IXTF/ucZz5i1Ot6Z96VtmCU1B6XXqhdjdUbDtz 0NeJKN9wUyFvkmIpscavkoZdou4eI45Pq4RBYuQQKNkIftBqgJL/IkeidNHIY6xu0923 HdCIplBQQInSuxKLYwj4ypgL+x7Se+Mb39oo7HIfpnhsGQHPtjOcX0XBmUek3icfs5Iv QE0Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=58kR9q2cEK71h1k3gjJqWvSSjyB6wHO/TzcyeXj29EE=; b=FK2ilazdPF/1Mdsh6K08CTyt8ABnOLiOFAmEqIjRxrSNByamQAH49VzcFs1J4dkm+g roJYsJF74vL0/eq1F40995j1FVNzrcb08gJFf7rXuNnQ3WTLTm3KlNwreJFFF3C8Fd/a PzyQmqzL7lrL9Jkjo/7ppFCQGELabedzt1BozocWzAFBNz1FImVzpjMBAmM4Zs/1aGH3 BhSNvgDmDTF9OonmiLoyJcAzDyrs/QQ+KyxCNoWPW5eY/C4a9KxFgk+dBBecF0fFEM5/ Y3u9XQcAb5iZ+mZ7SpZQ8X5Kb8GyqGgZGRt5FQ6j/2H5Uj6UCUwiq+2GDyj2TLzJ37A1 A4VA== X-Gm-Message-State: APjAAAXt2MBS+RqbzLUwu6dkytOCK8jSY5/Tn4LpLiHRwD4kGQwJtKRL miqjLmoviaMNSUYV4Cj+P521RzKbYMWRglglkWE= X-Google-Smtp-Source: APXvYqy7IWli4J99/GtHWlmk67tjn3St9EHIFkIuF00w+t5H+W72qc9qX567mpn+izofNc7WJNE5L/noJ788H1pmMnI= X-Received: by 2002:adf:ef0f:: with SMTP id e15mr10314169wro.312.1573247910957; Fri, 08 Nov 2019 13:18:30 -0800 (PST) MIME-Version: 1.0 References: <20191011160552.22907-1-vsementsov@virtuozzo.com> <20191011160552.22907-27-vsementsov@virtuozzo.com> In-Reply-To: <20191011160552.22907-27-vsementsov@virtuozzo.com> From: =?UTF-8?B?TWFyYy1BbmRyw6kgTHVyZWF1?= Date: Sat, 9 Nov 2019 01:18:18 +0400 Message-ID: Subject: Re: [RFC v5 026/126] python: add commit-per-subsystem.py To: Vladimir Sementsov-Ogievskiy Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:4864:20::443 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Ronnie Sahlberg , Jeff Cody , Jan Kiszka , Alberto Garcia , Hailiang Zhang , "open list:Block layer core" , Aleksandar Rikalo , Halil Pasic , =?UTF-8?Q?Herv=C3=A9_Poussineau?= , Anthony Perard , Samuel Thibault , Laszlo Ersek , Jason Wang , Laurent Vivier , Eduardo Habkost , Xie Changlong , Peter Lieven , "Dr. David Alan Gilbert" , Beniamino Galvani , Eric Auger , Alex Williamson , Stefan Hajnoczi , John Snow , Richard Henderson , Kevin Wolf , Andrew Jeffery , Chris Wulff , Subbaraya Sundeep , Michael Walle , "open list:sPAPR pseries" , Bastian Koppelmann , Igor Mammedov , Fam Zheng , Peter Maydell , sheepdog@lists.wpkg.org, Matthew Rosato , David Hildenbrand , Palmer Dabbelt , Eric Farman , Max Filippov , "Denis V. Lunev" , Hannes Reinecke , Stefano Stabellini , "Gonglei \(Arei\)" , Liu Yuan , Artyom Tarasenko , Thomas Huth , Amit Shah , Stefan Weil , Greg Kurz , Yuval Shaia , Qemu-s390x list , "open list:ARM" , Peter Chubb , =?UTF-8?Q?C=C3=A9dric_Le_Goater?= , Stafford Horne , qemu-riscv@nongnu.org, Cornelia Huck , Aleksandar Markovic , Aurelien Jarno , Paul Burton , Sagar Karandikar , Paul Durrant , Anthony Green , Gerd Hoffmann , "Edgar E. Iglesias" , Guan Xuetao , Ari Sundholm , Juan Quintela , Michael Roth , Christian Borntraeger , Joel Stanley , Jason Dillaman , Antony Pavlov , xen-devel@lists.xenproject.org, integration@gluster.org, =?UTF-8?Q?Philippe_Mathieu=2DDaud=C3=A9?= , "Richard W.M. Jones" , Andrew Baumann , Max Reitz , "Michael S. Tsirkin" , Mark Cave-Ayland , QEMU , Vincenzo Maffione , Marek Vasut , Markus Armbruster , Alistair Francis , Pavel Dovgalyuk , Giuseppe Lettieri , Luigi Rizzo , David Gibson , Tony Krowiak , =?UTF-8?Q?Daniel_P=2E_Berrang=C3=A9?= , Xiao Guangrong , Pierre Morel , Wen Congyang , Jean-Christophe Dubois , Paolo Bonzini , Stefan Berger Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Hi On Fri, Oct 11, 2019 at 9:11 PM Vladimir Sementsov-Ogievskiy wrote: > > Add script to automatically commit tree-wide changes per-subsystem. Oh interesting! I guess it could use a --help or a larger commit message to explain a bit what it does (I imagine from the rest of the series, but someone looking at the script without context may wonder;) You could also fix some pep8/pylint/pycodestyle > > Signed-off-by: Vladimir Sementsov-Ogievskiy > --- > > CC: Gerd Hoffmann > CC: "Gonglei (Arei)" > CC: Eduardo Habkost > CC: Igor Mammedov > CC: Laurent Vivier > CC: Amit Shah > CC: Kevin Wolf > CC: Max Reitz > CC: John Snow > CC: Ari Sundholm > CC: Pavel Dovgalyuk > CC: Paolo Bonzini > CC: Stefan Hajnoczi > CC: Fam Zheng > CC: Stefan Weil > CC: Ronnie Sahlberg > CC: Peter Lieven > CC: Eric Blake > CC: "Denis V. Lunev" > CC: Markus Armbruster > CC: Alberto Garcia > CC: Jason Dillaman > CC: Wen Congyang > CC: Xie Changlong > CC: Liu Yuan > CC: "Richard W.M. Jones" > CC: Jeff Cody > CC: "Marc-Andr=C3=A9 Lureau" > CC: "Daniel P. Berrang=C3=A9" > CC: Richard Henderson > CC: Greg Kurz > CC: "Michael S. Tsirkin" > CC: Marcel Apfelbaum > CC: Beniamino Galvani > CC: Peter Maydell > CC: "C=C3=A9dric Le Goater" > CC: Andrew Jeffery > CC: Joel Stanley > CC: Andrew Baumann > CC: "Philippe Mathieu-Daud=C3=A9" > CC: Antony Pavlov > CC: Jean-Christophe Dubois > CC: Peter Chubb > CC: Subbaraya Sundeep > CC: Eric Auger > CC: Alistair Francis > CC: "Edgar E. Iglesias" > CC: Stefano Stabellini > CC: Anthony Perard > CC: Paul Durrant > CC: Paul Burton > CC: Aleksandar Rikalo > CC: Chris Wulff > CC: Marek Vasut > CC: David Gibson > CC: Cornelia Huck > CC: Halil Pasic > CC: Christian Borntraeger > CC: "Herv=C3=A9 Poussineau" > CC: Xiao Guangrong > CC: Aurelien Jarno > CC: Aleksandar Markovic > CC: Mark Cave-Ayland > CC: Jason Wang > CC: Laszlo Ersek > CC: Yuval Shaia > CC: Palmer Dabbelt > CC: Sagar Karandikar > CC: Bastian Koppelmann > CC: David Hildenbrand > CC: Thomas Huth > CC: Eric Farman > CC: Matthew Rosato > CC: Hannes Reinecke > CC: Michael Walle > CC: Artyom Tarasenko > CC: Stefan Berger > CC: Samuel Thibault > CC: Alex Williamson > CC: Tony Krowiak > CC: Pierre Morel > CC: Michael Roth > CC: Hailiang Zhang > CC: Juan Quintela > CC: "Dr. David Alan Gilbert" > CC: Luigi Rizzo > CC: Giuseppe Lettieri > CC: Vincenzo Maffione > CC: Jan Kiszka > CC: Anthony Green > CC: Stafford Horne > CC: Guan Xuetao > CC: Max Filippov > CC: qemu-block@nongnu.org > CC: integration@gluster.org > CC: sheepdog@lists.wpkg.org > CC: qemu-arm@nongnu.org > CC: xen-devel@lists.xenproject.org > CC: qemu-ppc@nongnu.org > CC: qemu-s390x@nongnu.org > CC: qemu-riscv@nongnu.org > > python/commit-per-subsystem.py | 204 +++++++++++++++++++++++++++++++++ > 1 file changed, 204 insertions(+) > create mode 100755 python/commit-per-subsystem.py > > diff --git a/python/commit-per-subsystem.py b/python/commit-per-subsystem= .py > new file mode 100755 > index 0000000000..2ccf84cb15 > --- /dev/null > +++ b/python/commit-per-subsystem.py > @@ -0,0 +1,204 @@ > +#!/usr/bin/env python3 > +# > +# Copyright (c) 2019 Virtuozzo International GmbH > +# > +# This program is free software; you can redistribute it and/or modify > +# it under the terms of the GNU General Public License as published by > +# the Free Software Foundation; either version 2 of the License, or > +# (at your option) any later version. > +# > +# This program is distributed in the hope that it will be useful, > +# but WITHOUT ANY WARRANTY; without even the implied warranty of > +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > +# GNU General Public License for more details. > +# > +# You should have received a copy of the GNU General Public License > +# along with this program. If not, see . > +# > + > +import subprocess > +import sys > +import os > +import glob > + > + > +def git_add(pattern): > + subprocess.run(['git', 'add', pattern]) > + > + > +def git_commit(msg): > + subprocess.run(['git', 'commit', '-m', msg], capture_output=3DTrue) > + > + > +def git_changed_files(): > + ret =3D subprocess.check_output(['git', 'diff', '--name-only'], enco= ding=3D'utf-8').split('\n') > + if ret[-1] =3D=3D '': > + del ret[-1] > + return ret > + > + > +maintainers =3D sys.argv[1] > +message =3D sys.argv[2].strip() > + > +subsystem =3D None > + > +remap =3D { > + 'Block layer core': 'block', > + 'Block Jobs': 'block', > + 'Dirty Bitmaps': 'block', > + 'Block QAPI, monitor, command line': 'block', > + 'Block I/O path': 'block', > + 'Throttling infrastructure': 'block', > + 'Architecture support': 's390x', > + 'Guest CPU Cores (KVM)': 'kvm', > + 'Guest CPU Cores (Xen)': 'xen', > + 'Guest CPU cores (TCG)': 'tcg', > + 'Network Block Device (NBD)': 'nbd', > + 'Parallel NOR Flash devices': 'pflash', > + 'Firmware configuration (fw_cfg)': 'fw_cfg', > + 'Block SCSI subsystem': 'scsi', > + 'Network device backends': 'net', > + 'Netmap network backend': 'net', > + 'Host Memory Backends': 'hostmem', > + 'Cryptodev Backends': 'cryptodev', > + 'QEMU Guest Agent': 'qga', > + 'COLO Framework': 'colo', > + 'Command line option argument parsing': 'cmdline', > + 'Character device backends': 'chardev' > +} > + > + > +class Maintainers: > + def add(self, subsystem, path, mapper, mapper_name, glob_count=3D1): > + if subsystem in remap: > + subsystem =3D remap[subsystem] > + if subsystem not in self.subsystems: > + self.subsystems.append(subsystem) > + > + if path[-1] =3D=3D '/': > + path =3D path[:-1] > + > + if path in mapper: > + if mapper[path][1] =3D=3D glob_count: > + print('Warning: "{}" both in "{}" and "{}" in {} mapper = with ' > + 'same glob-count=3D{}. {} ignored for this path.'.= format( > + path, mapper[path][0], subsystem, mapper_name, g= lob_count, > + subsystem)) > + return > + if mapper[path][1] < glob_count: > + # silently ignore worse match > + return > + > + mapper[path] =3D (subsystem, glob_count) > + > + def __init__(self, file_name): > + self.map_file =3D {} > + self.map_glob_file =3D {} > + self.map_dir =3D {} > + self.map_glob_dir =3D {} > + self.map_unmaintained_dir =3D { > + 'python': ('python', 1), > + 'hw/misc': ('misc', 1) > + } > + self.subsystems =3D ['python', 'misc'] > + subsystem =3D None > + > + with open(file_name) as f: > + mode2 =3D False > + prevline =3D '' > + for line in f: > + line =3D line.rstrip() > + if not line: > + continue > + if len(line) >=3D 2 and line[1] =3D=3D ':': > + if line[0] =3D=3D 'F': > + fname =3D line[3:] > + if fname in ['*', '*/']: > + continue > + if os.path.isfile(fname): > + self.add(subsystem, fname, self.map_file, 'f= ile') > + elif os.path.isdir(fname): > + self.add(subsystem, fname, self.map_dir, 'di= r') > + else: > + paths =3D glob.glob(fname) > + if not paths: > + print('Warning: nothing corresponds to "= {}"'.format(fname)) > + continue > + > + n =3D len(paths) > + for f in paths: > + if os.path.isfile(f): > + self.add(subsystem, f, self.map_glob= _file, 'glob-file', n) > + else: > + assert os.path.isdir(f) > + self.add(subsystem, f, self.map_glob= _dir, 'glob-dir', n) > + elif line[:3] =3D=3D '---': > + subsystem =3D prevline > + if subsystem =3D=3D 'Devices': > + mode2 =3D True > + elif mode2: > + subsystem =3D line > + prevline =3D line > + > + def find_in_map_dir(self, file_name, mapper): > + while file_name !=3D '' and file_name not in mapper: > + file_name =3D os.path.dirname(file_name) > + > + return None if file_name =3D=3D '' else mapper[file_name][0] > + > + def find_in_map_file(self, file_name, mapper): > + if file_name in mapper: > + return mapper[file_name][0] > + > + def find_subsystem(self, file_name): > + s =3D self.find_in_map_file(file_name, self.map_file) > + if s is not None: > + return s > + > + s =3D self.find_in_map_file(file_name, self.map_glob_file) > + if s is not None: > + return s > + > + s =3D self.find_in_map_dir(file_name, self.map_dir) > + if s is not None: > + return s > + > + s =3D self.find_in_map_dir(file_name, self.map_glob_dir) > + if s is not None: > + return s > + > + s =3D self.find_in_map_dir(file_name, self.map_unmaintained_dir) > + if s is not None: > + return s > + > + self.subsystems.append(file_name) > + return file_name > + > + > +def commit(subsystem): > + msg =3D subsystem > + if msg in remap: > + msg =3D remap[msg] > + msg +=3D ': ' + message > + git_commit(msg) > + > +mnt =3D Maintainers(maintainers) > +res =3D {} > +for f in git_changed_files(): > + s =3D mnt.find_subsystem(f) > + if s in res: > + res[s].append(f) > + else: > + res[s] =3D [f] > + > +for s in mnt.subsystems: > + if s in res: > + print(s) > + for f in res[s]: > + print(' ', f) > + > +for s in mnt.subsystems: > + if s in res: > + for f in res[s]: > + git_add(f) > + commit(s) > -- > 2.21.0 > > --=20 Marc-Andr=C3=A9 Lureau