Re: [PATCH RESEND v2 06/16] check_whence: error on duplicate file entries

[Josh Boyer <jwboyer@kernel.org>]

On Wed, Mar 1, 2023 at 1:56 PM Emil Velikov via B4 Relay
<devnull+emil.l.velikov.gmail.com@kernel.org> wrote:
>
> From: Emil Velikov <emil.l.velikov@gmail.com>
>
> There's little point in copying (or compressing with later patches) the
> same files multiple times. So let's error out when duplicate entries are
> present.

I like this idea, but see my reply to patch 5.  There are cases where
the same firmware file *should* be duplicated because the driver using
it is different.  That would imply this would need to build out a
dictionary that can map files to drivers.  That seems over-complicated
for the fundamental goal of this check.

> Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
> ---
>  check_whence.py | 13 +++++++++++++
>  1 file changed, 13 insertions(+)
>
> diff --git a/check_whence.py b/check_whence.py
> index f347f0e..7ff21f6 100755
> --- a/check_whence.py
> +++ b/check_whence.py
> @@ -24,6 +24,14 @@ def list_whence():
>                          yield match.group(2)
>                          continue
>
> +def list_whence_files():
> +    with open('WHENCE', encoding='utf-8') as whence:
> +        for line in whence:
> +            match = re.match(r'File:\s*(.*)', line)
> +            if match:
> +                yield match.group(1).replace("\ ", " ")
> +                continue
> +
>  def list_git():
>      with os.popen('git ls-files') as git_files:
>          for line in git_files:
> @@ -32,12 +40,17 @@ def list_git():
>  def main():
>      ret = 0
>      whence_list = list(list_whence())
> +    whence_files = list(list_whence_files())
>      known_files = set(name for name in whence_list if not name.endswith('/')) | \
>                    set(['check_whence.py', 'configure', 'Makefile',
>                         'README', 'copy-firmware.sh', 'WHENCE'])
>      known_prefixes = set(name for name in whence_list if name.endswith('/'))
>      git_files = set(list_git())
>
> +    for name in set(fw for fw in whence_files if whence_files.count(fw) > 1):
> +        sys.stderr.write('E: %s listed in WHENCE twice\n' % name)
> +        ret = 1
> +

Perhaps we can add this check via a cmdline option that is run
occasionally to look for duplicates?  Then you could still keep
simpler code for anything that's operating on the files themselves.

josh

>      for name in sorted(list(known_files - git_files)):
>          sys.stderr.write('E: %s listed in WHENCE does not exist\n' % name)
>          ret = 1
>
> --
> 2.39.2
>