[PATCH v2 0/7] Fix Python string escapes

* [PATCH v2 0/7] Fix Python string escapes
@ 2023-09-12  6:07 Benjamin Gray
  2023-09-12  6:07 ` [PATCH v2 1/7] ia64: fix " Benjamin Gray
                   ` (6 more replies)
  0 siblings, 7 replies; 15+ messages in thread
From: Benjamin Gray @ 2023-09-12  6:07 UTC (permalink / raw)
  To: linux-ia64, linux-kernel, linux-doc, llvm, linux-pm, bpf,
	linux-kselftest
  Cc: Jonathan Corbet, Ian Abbott, H Hartley Sweeten,
	Nathan Chancellor, Nick Desaulniers, Tom Rix, Jan Kiszka,
	Kieran Bingham, Peter Zijlstra, Ingo Molnar,
	Arnaldo Carvalho de Melo, Mark Rutland, Alexander Shishkin,
	Jiri Olsa, Namhyung Kim, Ian Rogers, Adrian Hunter,
	linux-perf-users, Todd E Brandt, Andrii Nakryiko, Mykola Lysenko,
	Shuah Khan, Benjamin Gray

Changes from v1:
* Dropped some changes that were independently fixed[1]
* No longer separate the f strings to their own patch
* Use r strings when the value is a regular expression
* Updated verification script

In retrospect a script to find the instances and apply fixes isn't that
useful for review, so the attached script this time just looks for
differences in the AST. Apply the series and run the script, with
the two references to compare as arguments.

There are some intentional changes to the AST now though, as the r strings
turn '\t' from a single character tab into a backslash and 't' character
pair (similar for '\n'). This does not affect the correctness of the
regular expression though.

v1: https://lore.kernel.org/all/20230814060704.79655-1-bgray@linux.ibm.com/

[1]: https://lore.kernel.org/all/20230816122133.1231599-1-vishalc@linux.ibm.com/

---
#!/usr/bin/env python3

"""
Verify Python syntax trees are equivalent between two references
"""

import argparse
import ast
from pathlib import Path
import subprocess as sp

def read_file(path: Path, ref: str) -> str:
    return sp.run(f"git show {ref}:{path}", stdout=sp.PIPE, shell=True, encoding="utf-8", check=True).stdout

parser = argparse.ArgumentParser("Compare Python ASTs between revisions")
parser.add_argument("ref1", type=str, help="First revision to use")
parser.add_argument("ref2", type=str, help="Second revision to use")
args = parser.parse_args()

for pyfile in Path(".").glob("**/*.py"):
    try:
        ref1_content = read_file(pyfile, args.ref1)
        ref2_content = read_file(pyfile, args.ref2)
    except Exception as e:
        print(f"ERROR:{pyfile}: Failed to read ({e})")
        continue

    try:
        ref1_syntax = ast.parse(ref1_content, filename=pyfile)
        ref2_syntax = ast.parse(ref2_content, filename=pyfile)
    except SyntaxError as e:
        print(f"ERROR:{pyfile}: Failed to parse, is it Python3? ({e})")
        continue

    if ast.dump(ref1_syntax) != ast.dump(ref2_syntax):
        print(f"ERROR:{pyfile}: Revisions have different AST")
        cmd = f"diff <(git show {args.ref1}:{pyfile} | python -m ast) <(git show {args.ref2}:{pyfile} | python -m ast)"
        print(cmd)
        sp.run(cmd, shell=True)
        continue

Benjamin Gray (7):
  ia64: fix Python string escapes
  Documentation/sphinx: fix Python string escapes
  drivers/comedi: fix Python string escapes
  scripts: fix Python string escapes
  tools/perf: fix Python string escapes
  tools/power: fix Python string escapes
  selftests/bpf: fix Python string escapes

 Documentation/sphinx/cdomain.py               |  2 +-
 Documentation/sphinx/kernel_abi.py            |  2 +-
 Documentation/sphinx/kernel_feat.py           |  2 +-
 Documentation/sphinx/kerneldoc.py             |  2 +-
 Documentation/sphinx/maintainers_include.py   |  8 +++---
 arch/ia64/scripts/unwcheck.py                 |  2 +-
 .../ni_routing/tools/convert_csv_to_c.py      |  2 +-
 scripts/clang-tools/gen_compile_commands.py   |  2 +-
 scripts/gdb/linux/symbols.py                  |  2 +-
 tools/perf/pmu-events/jevents.py              |  2 +-
 .../scripts/python/arm-cs-trace-disasm.py     |  4 +--
 tools/perf/scripts/python/compaction-times.py |  2 +-
 .../scripts/python/exported-sql-viewer.py     |  4 +--
 tools/power/pm-graph/bootgraph.py             | 12 ++++-----
 .../selftests/bpf/test_bpftool_synctypes.py   | 26 +++++++++----------
 tools/testing/selftests/bpf/test_offload.py   |  2 +-
 16 files changed, 38 insertions(+), 38 deletions(-)

--
2.41.0

^ permalink raw reply	[flat|nested] 15+ messages in thread