All of lore.kernel.org
 help / color / mirror / Atom feed
From: daniel.sangorrin@toshiba.co.jp (daniel.sangorrin at toshiba.co.jp)
To: cip-dev@lists.cip-project.org
Subject: [cip-dev] [cip-kernel-sec 5/6] report_affected: add support for reporting on tags
Date: Wed, 10 Jul 2019 01:36:42 +0000	[thread overview]
Message-ID: <OSBPR01MB3077C550C531A1A3BA97C10BD0F00@OSBPR01MB3077.jpnprd01.prod.outlook.com> (raw)
In-Reply-To: <1561754773.21054.86.camel@codethink.co.uk>

> From: Ben Hutchings <ben.hutchings@codethink.co.uk>
> 
> On Tue, 2019-06-25 at 12:26 +0900, Daniel Sangorrin wrote:
> > Reporting on tags is useful for product engineers that
> > have shipped a kernel with a specific tag and need to know
> > which issues affect their product after some time.
> 
> I think this is a really important feature, but I'm not convinced about
> this implementation.

Glad to hear about the importance. Other members also expressed their interest.
 
> > Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
> > ---
> > ?scripts/report_affected.py | 60 ++++++++++++++++++++++++++++++++------
> > ?1 file changed, 51 insertions(+), 9 deletions(-)
> >
> > diff --git a/scripts/report_affected.py b/scripts/report_affected.py
> > index 7557dc8..32e9345 100755
> > --- a/scripts/report_affected.py
> > +++ b/scripts/report_affected.py
> > @@ -9,7 +9,9 @@
> > ?# Report issues affecting each stable branch.
> >
> > ?import argparse
> > +import copy
> > ?import subprocess
> > +import re
> >
> > ?import kernel_sec.branch
> > ?import kernel_sec.issue
> > @@ -23,10 +25,26 @@ def main(git_repo, remotes,
> > ?????????branches = []
> > ?????????for branch in live_branches:
> > ?????????????for name in branch_names:
> > +????????????????# there could be multiple tags for the same branch
> > +????????????????branch_copy = copy.deepcopy(branch)
> 
> This makes a lot of unnecessary copies!

Sorry about that. The new version only makes copies when necessary (one copy per branch name supplied by the user).


> 
> > +????????????????if name[0] == 'v':
> > +????????????????????# a stable tag, e.g. v4.4.92-cip11
> > +????????????????????branch_copy['tag'] = name
> > +????????????????????match = re.match(r'^v(\d+\.\d+).*', name)
> > +????????????????????if not match:
> > +????????????????????????raise ValueError('failed to parse tag %r' % name)
> > +????????????????????if 'cip' in name:
> > +????????????????????????name = 'linux-%s.y-cip' % match.group(1)
> > +????????????????????else:
> > +????????????????????????name = 'linux-%s.y' % match.group(1)
> 
> How about adding regexps for tags to the branch configuration?  Then
> here we could match tags to branches with
> re.matchall(branch['tag_regexp'], name).

OK, I have tried. Let me know if my implementation is what you had in mind.
I used re.match, not re.matchall (maybe you meant re.findall?). Let me know if there is a problem with that.
Tags that are not on a stable branch (e.g. 5.2 is in mainline but not in a stable branch) are ignored at the moment.

> 
> > +????????????????if '/' in name:
> 
> '/' is valid in a branch or tag name, so how about using ':' which is
> not?

Thanks, I had chosen '/' just for aesthetics (the final report uses : before displaying the issues).
Now fixed.

> 
> > +????????????????????# a possibly custom tag, e.g. product-v1
> > +????????????????????branch_copy['tag'] = name.split('/')[1]
> > +????????????????????name = name.split('/')[0]
> 
> It would be more Pythonic to write this as a tuple assginment.

I tried using tuples, but I'm not sure it is pythonic enough.
There is also a "tag = None" that may not be pythonic either.

> 
> > ?????????????????if name[0].isdigit():
> > ?????????????????????name = 'linux-%s.y' % name
> > -????????????????if branch['short_name'] == name:
> > -????????????????????branches.append(branch)
> > +????????????????if branch_copy['short_name'] == name:
> > +????????????????????branches.append(branch_copy)
> > ?????????if not branches:
> > ?????????????msg = "supplied branches didn't match any known branch"
> > ?????????????raise argparse.ArgumentError(None, msg)
> > @@ -40,6 +58,18 @@ def main(git_repo, remotes,
> >
> > ?????c_b_map = kernel_sec.branch.CommitBranchMap(git_repo, remotes, branches)
> >
> > +????# cache tag commits and set full_name to show the tag
> > +????tag_commits = {}
> > +????for branch in branches:
> > +????????if 'tag' in branch:
> > +????????????start = 'v' + branch['base_ver']
> > +????????????end = branch['tag']
> > +????????????for commit in kernel_sec.branch._get_commits(git_repo, end, start):
> 
> The leading '_' means private, so it shouldn't be called from here.  I
> think it could be made public but it needs a better name, maybe
> 'iter_rev_list' matching the underlying command.

Fixed.

> > +????????????????tag_commits.setdefault(end, []).append(commit)
> [...]
> 
> Suppose I tried to query 'v4.4'; then I think we get start == end ==
> 'v4.4' and this loop doesn't execute at all.  tag_commits['v4.4'] won't
> be defined at all, and the reporting loop will crash.

Good catch, I should have tested the edge cases sorry.

> So this could be simply:
> 
>                tag_commits[end] = list(
>                    kernel_sec.branch.iter_rev_list(git_repo, end, start))
> 
> But I think the collection type should also be changed from list to
> set, so that tests for membership will be fast.

Thanks, I finally used "set" although I didn't see big performance changes.

Thanks,
Daniel

> 
> Ben.
> 
> --
> Ben Hutchings, Software Developer                ?        Codethink Ltd
> https://www.codethink.co.uk/                 Dale House, 35 Dale Street
>                                      Manchester, M1 2HF, United Kingdom

  reply	other threads:[~2019-07-10  1:36 UTC|newest]

Thread overview: 13+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-06-25  3:26 [cip-dev] fixes and support for tags Daniel Sangorrin
2019-06-25  3:26 ` [cip-dev] [cip-kernel-sec 1/6] check_git_repo: add checks to the local repository Daniel Sangorrin
2019-06-25  3:26 ` [cip-dev] [cip-kernel-sec 2/6] prepare_remotes: helper script to prepare local repo Daniel Sangorrin
2019-06-25  3:26 ` [cip-dev] [cip-kernel-sec 3/6] report_affected: fix code when branches are specified Daniel Sangorrin
2019-06-28 20:09   ` Ben Hutchings
2019-07-10  1:28     ` daniel.sangorrin at toshiba.co.jp
2019-06-25  3:26 ` [cip-dev] [cip-kernel-sec 4/6] report_affected: check user supplied branch names Daniel Sangorrin
2019-06-28 20:12   ` Ben Hutchings
2019-07-10  1:29     ` daniel.sangorrin at toshiba.co.jp
2019-06-25  3:26 ` [cip-dev] [cip-kernel-sec 5/6] report_affected: add support for reporting on tags Daniel Sangorrin
2019-06-28 20:46   ` Ben Hutchings
2019-07-10  1:36     ` daniel.sangorrin at toshiba.co.jp [this message]
2019-06-25  3:26 ` [cip-dev] [cip-kernel-sec 6/6] pep8: fix pep8-related errors such as too long lines Daniel Sangorrin

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=OSBPR01MB3077C550C531A1A3BA97C10BD0F00@OSBPR01MB3077.jpnprd01.prod.outlook.com \
    --to=daniel.sangorrin@toshiba.co.jp \
    --cc=cip-dev@lists.cip-project.org \
    --subject='Re: [cip-dev] [cip-kernel-sec 5/6] report_affected: add support for reporting on tags' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.