[PATCH] pylint: Adjust how the output is produced

[PATCH] pylint: Adjust how the output is produced

[Simon Glass]

The current Makefile rule requires there to be a 'Module' line in the
pylint output, like this:

   ************* Module binman.fip_util

This line only appears if pylint has some comments about the module. We
cannot rely on it for naming.

Update the code to instead use the filename as the identifier for each
score, so rather than:

   multiplexed_log 7.49

we output:

   test_multiplexed_log.py 7.20

It is still easy to see which file the score relates to. In fact the new
naming is nicer since the full subdirectories are shown.

The avoids the problem where a module name is not produced, and the output
gets out of sync.

Regenerate pylint.base so we can start from the current baseline.

Signed-off-by: Simon Glass <sjg@chromium.org>
---

 Makefile            |   7 +-
 scripts/pylint.base | 441 +++++++++++++++++++++++---------------------
 2 files changed, 230 insertions(+), 218 deletions(-)

diff --git a/Makefile b/Makefile
index 10879f1f9cf..bb0c8d3d201 100644
--- a/Makefile
+++ b/Makefile
@@ -2269,9 +2269,10 @@ pylint:
 	$(Q)find tools test -name "*.py" \
 		| xargs -n1 -P$(shell nproc 2>/dev/null || echo 1) \
 			sh -c 'pylint --reports=y --exit-zero -f parseable --ignore-imports=yes $$@ > pylint.out/$$(echo $$@ | tr / _ | sed s/.py//)' _
-	$(Q)sed -n 's/Your code has been rated at \([-0-9.]*\).*/\1/p; s/\*\** Module \(.*\)/\1/p' pylint.out/* \
-		|sed '$!N;s/\n/ /' \
-		|sort > $(PYLINT_CUR)
+	$(Q)rm -f $(PYLINT_CUR)
+	$(Q)( cd pylint.out; for f in *; do \
+		sed -ne "s/Your code has been rated at \([-0-9.]*\).*/$$f \1/p" $$f; \
+	done ) | sort > $(PYLINT_CUR)
 	$(Q)base=$$(mktemp) cur=$$(mktemp); cut -d' ' -f1 $(PYLINT_BASE) >$$base; \
 		cut -d' ' -f1 $(PYLINT_CUR) >$$cur; \
 		comm -3 $$base $$cur > $(PYLINT_DIFF); \
diff --git a/scripts/pylint.base b/scripts/pylint.base
index cefdc235550..8c650f25729 100644
--- a/scripts/pylint.base
+++ b/scripts/pylint.base
@@ -1,215 +1,226 @@
-_testing 0.83
-atf_bl31 -6.00
-atf_fip 0.29
-binman.cbfs_util 8.38
-binman.cbfs_util_test 9.30
-binman.cmdline 9.09
-binman.control 4.92
-binman.elf 6.73
-binman.elf_test 5.41
-binman.entry 3.38
-binman.entry_test 5.34
-binman.fdt_test 3.23
-binman.fip_util 9.86
-binman.fip_util_test 9.75
-binman.fmap_util 6.88
-binman.ftest 7.46
-binman.image 7.05
-binman.image_test 4.48
-binman.main 5.00
-binman.setup 5.00
-binman.state 4.15
-blob -1.58
-blob_dtb -10.00
-blob_ext -19.09
-blob_ext_list -0.32
-blob_named_by_arg -7.78
-blob_phase -5.00
-buildman.board 7.82
-buildman.bsettings 1.71
-buildman.builder 6.91
-buildman.builderthread 7.39
-buildman.cmdline 9.04
-buildman.control 8.10
-buildman.func_test 7.18
-buildman.kconfiglib 7.49
-buildman.main 1.43
-buildman.test 6.17
-buildman.toolchain 6.55
-capsule_defs 5.00
-cbfs -1.44
-collection 2.67
-concurrencytest 7.26
-conftest -3.29
-conftest 1.88
-conftest 5.13
-conftest 6.56
-cros_ec_rw -6.00
-defs 6.67
-dtoc.dtb_platdata 7.90
-dtoc.fdt 4.50
-dtoc.fdt_util 6.70
-dtoc.main 7.78
-dtoc.setup 5.00
-dtoc.src_scan 8.91
-dtoc.test_dtoc 8.56
-dtoc.test_fdt 6.96
-dtoc.test_src_scan 9.43
-efivar 6.71
-endian-swap 9.29
-fdtmap -3.28
-files -7.43
-fill -6.43
-fit 5.32
-fmap -0.29
-fstest_defs 8.33
-fstest_helpers 4.29
-gbb -0.30
-genboardscfg 7.95
-image_header 5.77
-intel_cmc -12.50
-intel_descriptor 4.62
-intel_fit 0.00
-intel_fit_ptr 2.35
-intel_fsp -12.50
-intel_fsp_m -12.50
-intel_fsp_s -12.50
-intel_fsp_t -12.50
-intel_ifwi 2.71
-intel_me -12.50
-intel_mrc -10.00
-intel_refcode -10.00
-intel_vbt -12.50
-intel_vga -12.50
-microcode-tool 7.25
-mkimage 2.57
-moveconfig 8.32
-multiplexed_log 7.49
-opensbi -6.00
-patman 0.00
-patman.checkpatch 8.04
-patman.command 4.74
-patman.commit 3.25
-patman.control 8.14
-patman.cros_subprocess 7.56
-patman.func_test 8.14
-patman.get_maintainer 6.47
-patman.gitutil 5.62
-patman.main 8.23
-patman.patchstream 9.11
-patman.project 6.67
-patman.series 6.16
-patman.settings 5.89
-patman.setup 5.00
-patman.status 8.62
-patman.terminal 7.05
-patman.test_checkpatch 6.81
-patman.test_util 6.89
-patman.tools 4.31
-patman.tout 3.12
-powerpc_mpc85xx_bootpg_resetvec -10.00
-rkmux 6.90
-rmboard 7.76
-scp -6.00
-section 4.68
-sqfs_common 8.41
-test 8.18
-test_000_version 7.50
-test_ab 6.50
-test_abootimg 6.09
-test_authvar 8.93
-test_avb 5.52
-test_basic 0.60
-test_bind -2.99
-test_button 3.33
-test_capsule_firmware 3.89
-test_dfu 5.45
-test_dm 9.52
-test_efi_fit 8.16
-test_efi_loader 7.38
-test_efi_selftest 6.36
-test_env 7.15
-test_ext 0.00
-test_extension 2.14
-test_fit 6.83
-test_fit_ecdsa 7.94
-test_fit_hashes 7.70
-test_fpga 1.81
-test_fs_cmd 8.00
-test_gpio 6.09
-test_gpt 7.67
-test_handoff 5.00
-test_help 5.00
-test_hush_if_test 9.27
-test_log 8.64
-test_lsblk 8.00
-test_md 3.64
-test_mkdir 1.96
-test_mmc_rd 6.05
-test_mmc_wr 3.33
-test_net 6.84
-test_ofplatdata 5.71
-test_part 8.00
-test_pinmux 3.27
-test_pstore 2.31
-test_qfw 8.75
-test_sandbox_exit 6.50
-test_scp03 3.33
-test_sf 7.13
-test_shell_basics 9.58
-test_signed 8.38
-test_signed_intca 8.10
-test_sleep 7.78
-test_spl 2.22
-test_sqfs_load 7.46
-test_sqfs_ls 8.00
-test_stackprotector 5.71
-test_symlink 1.22
-test_tpm2 8.51
-test_ums 6.32
-test_unknown_cmd 5.00
-test_unlink 2.78
-test_unsigned 8.00
-test_ut 7.06
-test_vboot 6.00
-text -0.48
-u_boot -15.71
-u_boot_console_base 7.08
-u_boot_console_exec_attach 9.23
-u_boot_console_sandbox 8.06
-u_boot_dtb -12.22
-u_boot_dtb_with_ucode 0.39
-u_boot_elf -8.42
-u_boot_env 0.74
-u_boot_expanded -10.00
-u_boot_img -15.71
-u_boot_nodtb -15.71
-u_boot_spawn 7.65
-u_boot_spl -10.91
-u_boot_spl_bss_pad -9.29
-u_boot_spl_dtb -12.22
-u_boot_spl_elf -15.71
-u_boot_spl_expanded -9.09
-u_boot_spl_nodtb -10.91
-u_boot_spl_with_ucode_ptr -5.00
-u_boot_tpl -10.91
-u_boot_tpl_bss_pad -9.29
-u_boot_tpl_dtb -12.22
-u_boot_tpl_dtb_with_ucode -7.50
-u_boot_tpl_elf -15.71
-u_boot_tpl_expanded -9.09
-u_boot_tpl_nodtb -10.91
-u_boot_tpl_with_ucode_ptr -20.83
-u_boot_ucode 1.52
-u_boot_utils 6.94
-u_boot_with_ucode_ptr -0.71
-vblock -1.61
-vboot_evil 8.95
-vboot_forge 9.22
-x86_reset16 -15.71
-x86_reset16_spl -15.71
-x86_reset16_tpl -15.71
-x86_start16 -15.71
-x86_start16_spl -15.71
-x86_start16_tpl -15.71
-zynqmp_pm_cfg_obj_convert 6.67
+test_conftest.py 6.43
+test_multiplexed_log.py 7.20
+test_test.py 8.18
+test_tests_test_000_version.py 7.50
+test_tests_test_android_test_ab.py 6.50
+test_tests_test_android_test_abootimg.py 6.09
+test_tests_test_android_test_avb.py 5.52
+test_tests_test_bind.py -2.99
+test_tests_test_button.py 3.33
+test_tests_test_dfu.py 5.45
+test_tests_test_dm.py 9.52
+test_tests_test_efi_capsule_capsule_defs.py 5.00
+test_tests_test_efi_capsule_conftest.py 1.25
+test_tests_test_efi_capsule_test_capsule_firmware.py 3.89
+test_tests_test_efi_fit.py 7.59
+test_tests_test_efi_loader.py 7.38
+test_tests_test_efi_secboot_conftest.py -3.84
+test_tests_test_efi_secboot_defs.py 6.67
+test_tests_test_efi_secboot_test_authvar.py 8.93
+test_tests_test_efi_secboot_test_signed.py 8.38
+test_tests_test_efi_secboot_test_signed_intca.py 8.10
+test_tests_test_efi_secboot_test_unsigned.py 8.00
+test_tests_test_efi_selftest.py 6.36
+test_tests_test_env.py 7.08
+test_tests_test_extension.py 2.14
+test_tests_test_fit.py 6.20
+test_tests_test_fit_ecdsa.py 7.50
+test_tests_test_fit_hashes.py 7.70
+test_tests_test_fpga.py 1.81
+test_tests_test_fs_conftest.py 4.62
+test_tests_test_fs_fstest_defs.py 8.33
+test_tests_test_fs_fstest_helpers.py 4.29
+test_tests_test_fs_test_basic.py 0.40
+test_tests_test_fs_test_ext.py -0.25
+test_tests_test_fs_test_fs_cmd.py 8.00
+test_tests_test_fs_test_mkdir.py 1.96
+test_tests_test_fs_test_squashfs_sqfs_common.py 8.12
+test_tests_test_fs_test_squashfs_test_sqfs_load.py 7.12
+test_tests_test_fs_test_squashfs_test_sqfs_ls.py 8.00
+test_tests_test_fs_test_symlink.py 0.82
+test_tests_test_fs_test_unlink.py 2.22
+test_tests_test_gpio.py 6.09
+test_tests_test_gpt.py 7.67
+test_tests_test_handoff.py 5.00
+test_tests_test_help.py 5.00
+test_tests_test_hush_if_test.py 9.27
+test_tests_test_log.py 8.64
+test_tests_test_lsblk.py 8.00
+test_tests_test_md.py 3.64
+test_tests_test_mmc_rd.py 6.05
+test_tests_test_mmc_wr.py 3.33
+test_tests_test_net.py 6.84
+test_tests_test_ofplatdata.py 5.71
+test_tests_test_part.py 8.00
+test_tests_test_pinmux.py 3.27
+test_tests_test_pstore.py 2.31
+test_tests_test_qfw.py 8.75
+test_tests_test_sandbox_exit.py 6.50
+test_tests_test_scp03.py 3.33
+test_tests_test_sf.py 7.02
+test_tests_test_shell_basics.py 9.58
+test_tests_test_sleep.py 7.78
+test_tests_test_spl.py 2.22
+test_tests_test_stackprotector.py 5.71
+test_tests_test_tpm2.py 8.51
+test_tests_test_ums.py 6.32
+test_tests_test_unknown_cmd.py 5.00
+test_tests_test_ut.py 7.06
+test_tests_test_vboot.py 6.00
+test_tests_vboot_evil.py 8.95
+test_tests_vboot_forge.py 9.22
+test_u_boot_console_base.py 6.80
+test_u_boot_console_exec_attach.py 8.85
+test_u_boot_console_sandbox.py 7.22
+test_u_boot_spawn.py 7.39
+test_u_boot_utils.py 6.25
+tools_binman_bintool 8.59
+tools_binman_bintool_test 9.87
+tools_binman_btool__testing 6.09
+tools_binman_btool_cbfstool 7.39
+tools_binman_btool_fiptool 7.62
+tools_binman_btool_futility 6.96
+tools_binman_btool_ifwitool 3.33
+tools_binman_btool_lz4 5.93
+tools_binman_btool_lzma_alone 6.67
+tools_binman_btool_mkimage 6.79
+tools_binman_cbfs_util 7.77
+tools_binman_cbfs_util_test 9.27
+tools_binman_cmdline 9.03
+tools_binman_comp_util 6.25
+tools_binman_control 4.60
+tools_binman_elf 6.42
+tools_binman_elf_test 5.41
+tools_binman_entry 2.48
+tools_binman_entry_test 5.34
+tools_binman_etype__testing 0.83
+tools_binman_etype_atf_bl31 -6.00
+tools_binman_etype_atf_fip 0.29
+tools_binman_etype_blob -1.58
+tools_binman_etype_blob_dtb -10.00
+tools_binman_etype_blob_ext -19.09
+tools_binman_etype_blob_ext_list 0.00
+tools_binman_etype_blob_named_by_arg -7.78
+tools_binman_etype_blob_phase -5.00
+tools_binman_etype_cbfs -1.44
+tools_binman_etype_collection 2.33
+tools_binman_etype_cros_ec_rw -6.00
+tools_binman_etype_fdtmap -3.61
+tools_binman_etype_files -7.43
+tools_binman_etype_fill -6.43
+tools_binman_etype_fit 5.18
+tools_binman_etype_fmap -0.59
+tools_binman_etype_gbb 0.83
+tools_binman_etype_image_header 5.58
+tools_binman_etype_intel_cmc -12.50
+tools_binman_etype_intel_descriptor 4.62
+tools_binman_etype_intel_fit 0.00
+tools_binman_etype_intel_fit_ptr 2.35
+tools_binman_etype_intel_fsp -12.50
+tools_binman_etype_intel_fsp_m -12.50
+tools_binman_etype_intel_fsp_s -12.50
+tools_binman_etype_intel_fsp_t -12.50
+tools_binman_etype_intel_ifwi 2.88
+tools_binman_etype_intel_me -12.50
+tools_binman_etype_intel_mrc -10.00
+tools_binman_etype_intel_refcode -10.00
+tools_binman_etype_intel_vbt -12.50
+tools_binman_etype_intel_vga -12.50
+tools_binman_etype_mkimage 2.31
+tools_binman_etype_opensbi -6.00
+tools_binman_etype_powerpc_mpc85xx_bootpg_resetvec -10.00
+tools_binman_etype_scp -6.00
+tools_binman_etype_section 4.12
+tools_binman_etype_text -0.48
+tools_binman_etype_u_boot -15.71
+tools_binman_etype_u_boot_dtb -12.22
+tools_binman_etype_u_boot_dtb_with_ucode 0.39
+tools_binman_etype_u_boot_elf -8.42
+tools_binman_etype_u_boot_env 0.74
+tools_binman_etype_u_boot_expanded -10.00
+tools_binman_etype_u_boot_img -15.71
+tools_binman_etype_u_boot_nodtb -15.71
+tools_binman_etype_u_boot_spl -10.91
+tools_binman_etype_u_boot_spl_bss_pad -9.29
+tools_binman_etype_u_boot_spl_dtb -12.22
+tools_binman_etype_u_boot_spl_elf -15.71
+tools_binman_etype_u_boot_spl_expanded -9.09
+tools_binman_etype_u_boot_spl_nodtb -10.91
+tools_binman_etype_u_boot_spl_with_ucode_ptr -5.00
+tools_binman_etype_u_boot_tpl -10.91
+tools_binman_etype_u_boot_tpl_bss_pad -9.29
+tools_binman_etype_u_boot_tpl_dtb -12.22
+tools_binman_etype_u_boot_tpl_dtb_with_ucode -7.50
+tools_binman_etype_u_boot_tpl_elf -15.71
+tools_binman_etype_u_boot_tpl_expanded -9.09
+tools_binman_etype_u_boot_tpl_nodtb -10.91
+tools_binman_etype_u_boot_tpl_with_ucode_ptr -20.83
+tools_binman_etype_u_boot_ucode 1.52
+tools_binman_etype_u_boot_with_ucode_ptr -0.71
+tools_binman_etype_vblock 0.00
+tools_binman_etype_x86_reset16 -15.71
+tools_binman_etype_x86_reset16_spl -15.71
+tools_binman_etype_x86_reset16_tpl -15.71
+tools_binman_etype_x86_start16 -15.71
+tools_binman_etype_x86_start16_spl -15.71
+tools_binman_etype_x86_start16_tpl -15.71
+tools_binman_fdt_test 3.23
+tools_binman_fip_util 9.85
+tools_binman_fip_util_test 10.00
+tools_binman_fmap_util 6.67
+tools_binman_ftest 7.39
+tools_binman_image 6.48
+tools_binman_image_test 4.48
+tools_binman_main 4.29
+tools_binman_setup 5.00
+tools_binman_state 3.43
+tools_buildman_board 7.11
+tools_buildman_bsettings 0.98
+tools_buildman_builder 6.55
+tools_buildman_builderthread 7.35
+tools_buildman_cmdline 8.85
+tools_buildman_control 7.04
+tools_buildman_func_test 6.38
+tools_buildman_kconfiglib 7.48
+tools_buildman_main 1.43
+tools_buildman_test 6.10
+tools_buildman_toolchain 5.62
+tools_concurrencytest_concurrencytest 6.77
+tools_dtoc_dtb_platdata 7.82
+tools_dtoc_fdt 3.47
+tools_dtoc_fdt_util 5.00
+tools_dtoc_main 7.33
+tools_dtoc_setup 5.00
+tools_dtoc_src_scan 8.75
+tools_dtoc_test_dtoc 8.54
+tools_dtoc_test_fdt 6.92
+tools_dtoc_test_src_scan 9.43
+tools_efivar 6.71
+tools_endian-swap 8.93
+tools_genboardscfg 7.27
+tools_microcode-tool 7.19
+tools_moveconfig 8.11
+tools_patman___init__ 0.00
+tools_patman_checkpatch 7.61
+tools_patman_command 4.23
+tools_patman_commit 2.75
+tools_patman_control 8.14
+tools_patman_cros_subprocess 7.41
+tools_patman_func_test 7.87
+tools_patman_get_maintainer 4.71
+tools_patman_gitutil 4.58
+tools_patman_main 8.23
+tools_patman_patchstream 9.04
+tools_patman_project 3.33
+tools_patman_series 5.95
+tools_patman_settings 5.63
+tools_patman_setup 5.00
+tools_patman_status 8.43
+tools_patman_terminal 6.29
+tools_patman_test_checkpatch 6.81
+tools_patman_test_util 6.51
+tools_patman_tools 3.98
+tools_patman_tout 2.97
+tools_rkmux 6.76
+tools_rmboard 7.76
+tools_zynqmp_pm_cfg_obj_convert 6.67
-- 
2.35.0.rc2.247.g8bbb082509-goog

Re: [PATCH] pylint: Adjust how the output is produced

[Tom Rini]

[-- Attachment #1: Type: text/plain, Size: 1022 bytes --]

On Fri, Jan 28, 2022 at 09:31:07PM -0700, Simon Glass wrote:

> The current Makefile rule requires there to be a 'Module' line in the
> pylint output, like this:
> 
>    ************* Module binman.fip_util
> 
> This line only appears if pylint has some comments about the module. We
> cannot rely on it for naming.
> 
> Update the code to instead use the filename as the identifier for each
> score, so rather than:
> 
>    multiplexed_log 7.49
> 
> we output:
> 
>    test_multiplexed_log.py 7.20
> 
> It is still easy to see which file the score relates to. In fact the new
> naming is nicer since the full subdirectories are shown.
> 
> The avoids the problem where a module name is not produced, and the output
> gets out of sync.
> 
> Regenerate pylint.base so we can start from the current baseline.
> 
> Signed-off-by: Simon Glass <sjg@chromium.org>

That does make it easier to see where a score comes from, thanks.  I'll
apply this soon if there's no further comments.

-- 
Tom

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 659 bytes --]

Re: [PATCH] pylint: Adjust how the output is produced

[Tom Rini]

[-- Attachment #1: Type: text/plain, Size: 935 bytes --]

On Fri, Jan 28, 2022 at 09:31:07PM -0700, Simon Glass wrote:

> The current Makefile rule requires there to be a 'Module' line in the
> pylint output, like this:
> 
>    ************* Module binman.fip_util
> 
> This line only appears if pylint has some comments about the module. We
> cannot rely on it for naming.
> 
> Update the code to instead use the filename as the identifier for each
> score, so rather than:
> 
>    multiplexed_log 7.49
> 
> we output:
> 
>    test_multiplexed_log.py 7.20
> 
> It is still easy to see which file the score relates to. In fact the new
> naming is nicer since the full subdirectories are shown.
> 
> The avoids the problem where a module name is not produced, and the output
> gets out of sync.
> 
> Regenerate pylint.base so we can start from the current baseline.
> 
> Signed-off-by: Simon Glass <sjg@chromium.org>

Applied to u-boot/master, thanks!

-- 
Tom

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 659 bytes --]