[PATCH 0/5] doc: Improve documentation for testing

[PATCH 0/5] doc: Improve documentation for testing

[Simon Glass]

This creates an introduction section for testing so that it is easier to
figure out what is going on.

It also moves the moveconfig docs to rST and brings in an old patch that
was rejected at the time.


Simon Glass (5):
  doc: Create an intro section for testing
  doc: Move coccinelle into its own section
  doc: Add docs for the moveconfig tool
  doc: Fix up outdated moveconfig docs
  moveconfig: Update to newer kconfiglib

 doc/develop/index.rst      |  10 +-
 doc/develop/moveconfig.rst | 282 +++++++++++++++++++++++++++++++++
 doc/develop/testing.rst    |   6 +-
 tools/moveconfig.py        | 315 ++-----------------------------------
 4 files changed, 308 insertions(+), 305 deletions(-)
 create mode 100644 doc/develop/moveconfig.rst

-- 
2.32.0.402.g57bb445576-goog

[PATCH 1/5] doc: Create an intro section for testing

[Simon Glass]

At present this information is hidden away. Make it more visible by
putting it first, in an intro section.

Signed-off-by: Simon Glass <sjg@chromium.org>
---

 doc/develop/index.rst   | 2 +-
 doc/develop/testing.rst | 6 ++++--
 2 files changed, 5 insertions(+), 3 deletions(-)

diff --git a/doc/develop/index.rst b/doc/develop/index.rst
index 3edffbc6373..901d39f4564 100644
--- a/doc/develop/index.rst
+++ b/doc/develop/index.rst
@@ -40,8 +40,8 @@ Testing
 .. toctree::
    :maxdepth: 1
 
-   coccinelle
    testing
+   coccinelle
    py_testing
    tests_writing
    tests_sandbox
diff --git a/doc/develop/testing.rst b/doc/develop/testing.rst
index ced13ac8bb4..1abe4d7f0f0 100644
--- a/doc/develop/testing.rst
+++ b/doc/develop/testing.rst
@@ -1,5 +1,7 @@
-Testing in U-Boot
-=================
+.. SPDX-License-Identifier: GPL-2.0+
+
+Introduction to testing
+=======================
 
 U-Boot has a large amount of code. This file describes how this code is
 tested and what tests you should write when adding a new feature.
-- 
2.32.0.402.g57bb445576-goog

Re: [PATCH 1/5] doc: Create an intro section for testing

[Heinrich Schuchardt]

On 7/22/21 5:35 AM, Simon Glass wrote:
> At present this information is hidden away. Make it more visible by
> putting it first, in an intro section.
>
> Signed-off-by: Simon Glass <sjg@chromium.org>
> ---
>
>   doc/develop/index.rst   | 2 +-
>   doc/develop/testing.rst | 6 ++++--
>   2 files changed, 5 insertions(+), 3 deletions(-)
>
> diff --git a/doc/develop/index.rst b/doc/develop/index.rst
> index 3edffbc6373..901d39f4564 100644
> --- a/doc/develop/index.rst
> +++ b/doc/develop/index.rst
> @@ -40,8 +40,8 @@ Testing
>   .. toctree::
>      :maxdepth: 1
>
> -   coccinelle
>      testing
> +   coccinelle

This change seems to be unrelated.

Otherwise

Reviewed-by: Heinrich Schuchardt <xypron.glpk@gmx.de>

>      py_testing
>      tests_writing
>      tests_sandbox
> diff --git a/doc/develop/testing.rst b/doc/develop/testing.rst
> index ced13ac8bb4..1abe4d7f0f0 100644
> --- a/doc/develop/testing.rst
> +++ b/doc/develop/testing.rst
> @@ -1,5 +1,7 @@
> -Testing in U-Boot
> -=================
> +.. SPDX-License-Identifier: GPL-2.0+
> +
> +Introduction to testing
> +=======================
>
>   U-Boot has a large amount of code. This file describes how this code is
>   tested and what tests you should write when adding a new feature.
>

Re: [PATCH 1/5] doc: Create an intro section for testing

[Simon Glass]

Hi Heinrich,

On Wed, 21 Jul 2021 at 23:52, Heinrich Schuchardt <xypron.glpk@gmx.de> wrote:
>
> On 7/22/21 5:35 AM, Simon Glass wrote:
> > At present this information is hidden away. Make it more visible by
> > putting it first, in an intro section.
> >
> > Signed-off-by: Simon Glass <sjg@chromium.org>
> > ---
> >
> >   doc/develop/index.rst   | 2 +-
> >   doc/develop/testing.rst | 6 ++++--
> >   2 files changed, 5 insertions(+), 3 deletions(-)
> >
> > diff --git a/doc/develop/index.rst b/doc/develop/index.rst
> > index 3edffbc6373..901d39f4564 100644
> > --- a/doc/develop/index.rst
> > +++ b/doc/develop/index.rst
> > @@ -40,8 +40,8 @@ Testing
> >   .. toctree::
> >      :maxdepth: 1
> >
> > -   coccinelle
> >      testing
> > +   coccinelle
>
> This change seems to be unrelated.

No, the point is to put the introduction to testing first, so I moved it up.

>
> Otherwise
>
> Reviewed-by: Heinrich Schuchardt <xypron.glpk@gmx.de>
>
> >      py_testing
> >      tests_writing
> >      tests_sandbox
> > diff --git a/doc/develop/testing.rst b/doc/develop/testing.rst
> > index ced13ac8bb4..1abe4d7f0f0 100644
> > --- a/doc/develop/testing.rst
> > +++ b/doc/develop/testing.rst
> > @@ -1,5 +1,7 @@
> > -Testing in U-Boot
> > -=================
> > +.. SPDX-License-Identifier: GPL-2.0+
> > +
> > +Introduction to testing
> > +=======================
> >
> >   U-Boot has a large amount of code. This file describes how this code is
> >   tested and what tests you should write when adding a new feature.
> >
>

Regards,
Simon

Re: [PATCH 1/5] doc: Create an intro section for testing

[Simon Glass]

Hi Heinrich,

On Wed, 21 Jul 2021 at 23:52, Heinrich Schuchardt <xypron.glpk@gmx.de> wrote:
>
> On 7/22/21 5:35 AM, Simon Glass wrote:
> > At present this information is hidden away. Make it more visible by
> > putting it first, in an intro section.
> >
> > Signed-off-by: Simon Glass <sjg@chromium.org>
> > ---
> >
> >   doc/develop/index.rst   | 2 +-
> >   doc/develop/testing.rst | 6 ++++--
> >   2 files changed, 5 insertions(+), 3 deletions(-)
> >
Applied to u-boot-dm, thanks!

[PATCH 2/5] doc: Move coccinelle into its own section

[Simon Glass]

This tool has nothing to do with testing. It is for refactoring code
automatically using a 'semantic patch' tool.

Create a new section for 'refactoring' and move it into there. It is
likely that other topics may fall under the same heading, such as
using moveconfig and search/replace tools.

Signed-off-by: Simon Glass <sjg@chromium.org>
---
This patch was submitted previously:

http://patchwork.ozlabs.org/project/uboot/patch/20210304135118.643277-38-sjg@chromium.org/

It was rejected, but I think that was incorrect.

 doc/develop/index.rst | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/doc/develop/index.rst b/doc/develop/index.rst
index 901d39f4564..2bd7c4a1a3a 100644
--- a/doc/develop/index.rst
+++ b/doc/develop/index.rst
@@ -41,7 +41,14 @@ Testing
    :maxdepth: 1
 
    testing
-   coccinelle
    py_testing
    tests_writing
    tests_sandbox
+
+Refactoring
+-----------
+
+.. toctree::
+   :maxdepth: 1
+
+   coccinelle
-- 
2.32.0.402.g57bb445576-goog

Re: [PATCH 2/5] doc: Move coccinelle into its own section

[Heinrich Schuchardt]

On 7/22/21 5:35 AM, Simon Glass wrote:
> This tool has nothing to do with testing. It is for refactoring code
> automatically using a 'semantic patch' tool.
>
> Create a new section for 'refactoring' and move it into there. It is
> likely that other topics may fall under the same heading, such as
> using moveconfig and search/replace tools.
>
> Signed-off-by: Simon Glass <sjg@chromium.org>
> ---
> This patch was submitted previously:
>
> http://patchwork.ozlabs.org/project/uboot/patch/20210304135118.643277-38-sjg@chromium.org/
>
> It was rejected, but I think that was incorrect.
>
>   doc/develop/index.rst | 9 ++++++++-
>   1 file changed, 8 insertions(+), 1 deletion(-)
>
> diff --git a/doc/develop/index.rst b/doc/develop/index.rst
> index 901d39f4564..2bd7c4a1a3a 100644
> --- a/doc/develop/index.rst
> +++ b/doc/develop/index.rst
> @@ -41,7 +41,14 @@ Testing
>      :maxdepth: 1
>
>      testing
> -   coccinelle
>      py_testing
>      tests_writing
>      tests_sandbox
> +
> +Refactoring
> +-----------

Coccinelle is used to detect common coding issues (e.g. using 0 instead
of NULL for pointers). How does this relate to refactoring?

Best regards

Heinrich

> +
> +.. toctree::
> +   :maxdepth: 1
> +
> +   coccinelle
>

Re: [PATCH 2/5] doc: Move coccinelle into its own section

[Simon Glass]

Hi Heinrich,

On Wed, 21 Jul 2021 at 23:53, Heinrich Schuchardt <xypron.glpk@gmx.de> wrote:
>
> On 7/22/21 5:35 AM, Simon Glass wrote:
> > This tool has nothing to do with testing. It is for refactoring code
> > automatically using a 'semantic patch' tool.
> >
> > Create a new section for 'refactoring' and move it into there. It is
> > likely that other topics may fall under the same heading, such as
> > using moveconfig and search/replace tools.
> >
> > Signed-off-by: Simon Glass <sjg@chromium.org>
> > ---
> > This patch was submitted previously:
> >
> > http://patchwork.ozlabs.org/project/uboot/patch/20210304135118.643277-38-sjg@chromium.org/
> >
> > It was rejected, but I think that was incorrect.
> >
> >   doc/develop/index.rst | 9 ++++++++-
> >   1 file changed, 8 insertions(+), 1 deletion(-)
> >
> > diff --git a/doc/develop/index.rst b/doc/develop/index.rst
> > index 901d39f4564..2bd7c4a1a3a 100644
> > --- a/doc/develop/index.rst
> > +++ b/doc/develop/index.rst
> > @@ -41,7 +41,14 @@ Testing
> >      :maxdepth: 1
> >
> >      testing
> > -   coccinelle
> >      py_testing
> >      tests_writing
> >      tests_sandbox
> > +
> > +Refactoring
> > +-----------
>
> Coccinelle is used to detect common coding issues (e.g. using 0 instead
> of NULL for pointers). How does this relate to refactoring?

Please read the docs in there. You will see a lot of discussion about
semantic patch (spatch). We use coccinelle to do global renames and
refactors.

Regards,
Simon

Re: [PATCH 2/5] doc: Move coccinelle into its own section

[Simon Glass]

Hi Heinrich,

On Wed, 21 Jul 2021 at 23:53, Heinrich Schuchardt <xypron.glpk@gmx.de> wrote:
>
> On 7/22/21 5:35 AM, Simon Glass wrote:
> > This tool has nothing to do with testing. It is for refactoring code
> > automatically using a 'semantic patch' tool.
> >
> > Create a new section for 'refactoring' and move it into there. It is
> > likely that other topics may fall under the same heading, such as
> > using moveconfig and search/replace tools.
> >
> > Signed-off-by: Simon Glass <sjg@chromium.org>
> > ---
> > This patch was submitted previously:
> >
> > http://patchwork.ozlabs.org/project/uboot/patch/20210304135118.643277-38-sjg@chromium.org/
> >
> > It was rejected, but I think that was incorrect.
> >
> >   doc/develop/index.rst | 9 ++++++++-
> >   1 file changed, 8 insertions(+), 1 deletion(-)
> >
Applied to u-boot-dm, thanks!

[PATCH 3/5] doc: Add docs for the moveconfig tool

[Simon Glass]

Move these docs into htmldocs so they can be read there.

Signed-off-by: Simon Glass <sjg@chromium.org>
---

 doc/develop/index.rst      |   1 +
 doc/develop/moveconfig.rst | 296 +++++++++++++++++++++++++++++++++++++
 tools/moveconfig.py        | 291 +-----------------------------------
 3 files changed, 298 insertions(+), 290 deletions(-)
 create mode 100644 doc/develop/moveconfig.rst

diff --git a/doc/develop/index.rst b/doc/develop/index.rst
index 2bd7c4a1a3a..3b8a6144ff9 100644
--- a/doc/develop/index.rst
+++ b/doc/develop/index.rst
@@ -52,3 +52,4 @@ Refactoring
    :maxdepth: 1
 
    coccinelle
+   moveconfig
diff --git a/doc/develop/moveconfig.rst b/doc/develop/moveconfig.rst
new file mode 100644
index 00000000000..aaa155e8c70
--- /dev/null
+++ b/doc/develop/moveconfig.rst
@@ -0,0 +1,296 @@
+.. SPDX-License-Identifier: GPL-2.0+
+
+moveconfig
+==========
+
+Since Kconfig was introduced to U-Boot, we have worked on moving
+config options from headers to Kconfig (defconfig).
+
+This tool intends to help this tremendous work.
+
+Installing
+----------
+
+You may need to install 'python3-asteval' for the 'asteval' module.
+
+Usage
+-----
+
+First, you must edit the Kconfig to add the menu entries for the configs
+you are moving.
+
+Then run this tool giving CONFIG names you want to move.
+For example, if you want to move CONFIG_CMD_USB and CONFIG_SYS_TEXT_BASE,
+simply type as follows::
+
+  $ tools/moveconfig.py CONFIG_CMD_USB CONFIG_SYS_TEXT_BASE
+
+The tool walks through all the defconfig files and move the given CONFIGs.
+
+The log is also displayed on the terminal.
+
+The log is printed for each defconfig as follows::
+
+  <defconfig_name>
+     <action1>
+     <action2>
+     <action3>
+     ...
+
+`<defconfig_name>` is the name of the defconfig.
+
+`<action*>` shows what the tool did for that defconfig.
+It looks like one of the following:
+
+ - Move 'CONFIG\_... '
+   This config option was moved to the defconfig
+
+ - CONFIG\_... is not defined in Kconfig.  Do nothing.
+   The entry for this CONFIG was not found in Kconfig.  The option is not
+   defined in the config header, either.  So, this case can be just skipped.
+
+ - CONFIG\_... is not defined in Kconfig (suspicious).  Do nothing.
+   This option is defined in the config header, but its entry was not found
+   in Kconfig.
+   There are two common cases:
+
+     - You forgot to create an entry for the CONFIG before running
+       this tool, or made a typo in a CONFIG passed to this tool.
+     - The entry was hidden due to unmet 'depends on'.
+
+   The tool does not know if the result is reasonable, so please check it
+   manually.
+
+ - 'CONFIG\_...' is the same as the define in Kconfig.  Do nothing.
+   The define in the config header matched the one in Kconfig.
+   We do not need to touch it.
+
+ - Compiler is missing.  Do nothing.
+   The compiler specified for this architecture was not found
+   in your PATH environment.
+   (If -e option is passed, the tool exits immediately.)
+
+ - Failed to process.
+   An error occurred during processing this defconfig.  Skipped.
+   (If -e option is passed, the tool exits immediately on error.)
+
+Finally, you will be asked, Clean up headers? [y/n]:
+
+If you say 'y' here, the unnecessary config defines are removed
+from the config headers (include/configs/\*.h).
+It just uses the regex method, so you should not rely on it.
+Just in case, please do 'git diff' to see what happened.
+
+
+How does it work?
+-----------------
+
+This tool runs configuration and builds include/autoconf.mk for every
+defconfig.  The config options defined in Kconfig appear in the .config
+file (unless they are hidden because of unmet dependency.)
+On the other hand, the config options defined by board headers are seen
+in include/autoconf.mk.  The tool looks for the specified options in both
+of them to decide the appropriate action for the options.  If the given
+config option is found in the .config, but its value does not match the
+one from the board header, the config option in the .config is replaced
+with the define in the board header.  Then, the .config is synced by
+"make savedefconfig" and the defconfig is updated with it.
+
+For faster processing, this tool handles multi-threading.  It creates
+separate build directories where the out-of-tree build is run.  The
+temporary build directories are automatically created and deleted as
+needed.  The number of threads are chosen based on the number of the CPU
+cores of your system although you can change it via -j (--jobs) option.
+
+
+Toolchains
+----------
+
+Appropriate toolchain are necessary to generate include/autoconf.mk
+for all the architectures supported by U-Boot.  Most of them are available
+at the kernel.org site, some are not provided by kernel.org. This tool uses
+the same tools as buildman, so see that tool for setup (e.g. --fetch-arch).
+
+
+Tips and trips
+--------------
+
+To sync only X86 defconfigs::
+
+   ./tools/moveconfig.py -s -d <(grep -l X86 configs/*)
+
+or::
+
+   grep -l X86 configs/* | ./tools/moveconfig.py -s -d -
+
+To process CONFIG_CMD_FPGAD only for a subset of configs based on path match::
+
+   ls configs/{hrcon*,iocon*,strider*} | \
+       ./tools/moveconfig.py -Cy CONFIG_CMD_FPGAD -d -
+
+
+Finding implied CONFIGs
+-----------------------
+
+Some CONFIG options can be implied by others and this can help to reduce
+the size of the defconfig files. For example, CONFIG_X86 implies
+CONFIG_CMD_IRQ, so we can put 'imply CMD_IRQ' under 'config X86' and
+all x86 boards will have that option, avoiding adding CONFIG_CMD_IRQ to
+each of the x86 defconfig files.
+
+This tool can help find such configs. To use it, first build a database::
+
+    ./tools/moveconfig.py -b
+
+Then try to query it::
+
+    ./tools/moveconfig.py -i CONFIG_CMD_IRQ
+    CONFIG_CMD_IRQ found in 311/2384 defconfigs
+    44 : CONFIG_SYS_FSL_ERRATUM_IFC_A002769
+    41 : CONFIG_SYS_FSL_ERRATUM_A007075
+    31 : CONFIG_SYS_FSL_DDR_VER_44
+    28 : CONFIG_ARCH_P1010
+    28 : CONFIG_SYS_FSL_ERRATUM_P1010_A003549
+    28 : CONFIG_SYS_FSL_ERRATUM_SEC_A003571
+    28 : CONFIG_SYS_FSL_ERRATUM_IFC_A003399
+    25 : CONFIG_SYS_FSL_ERRATUM_A008044
+    22 : CONFIG_ARCH_P1020
+    21 : CONFIG_SYS_FSL_DDR_VER_46
+    20 : CONFIG_MAX_PIRQ_LINKS
+    20 : CONFIG_HPET_ADDRESS
+    20 : CONFIG_X86
+    20 : CONFIG_PCIE_ECAM_SIZE
+    20 : CONFIG_IRQ_SLOT_COUNT
+    20 : CONFIG_I8259_PIC
+    20 : CONFIG_CPU_ADDR_BITS
+    20 : CONFIG_RAMBASE
+    20 : CONFIG_SYS_FSL_ERRATUM_A005871
+    20 : CONFIG_PCIE_ECAM_BASE
+    20 : CONFIG_X86_TSC_TIMER
+    20 : CONFIG_I8254_TIMER
+    20 : CONFIG_CMD_GETTIME
+    19 : CONFIG_SYS_FSL_ERRATUM_A005812
+    18 : CONFIG_X86_RUN_32BIT
+    17 : CONFIG_CMD_CHIP_CONFIG
+    ...
+
+This shows a list of config options which might imply CONFIG_CMD_EEPROM along
+with how many defconfigs they cover. From this you can see that CONFIG_X86
+implies CONFIG_CMD_EEPROM. Therefore, instead of adding CONFIG_CMD_EEPROM to
+the defconfig of every x86 board, you could add a single imply line to the
+Kconfig file:
+
+    config X86
+        bool "x86 architecture"
+        ...
+        imply CMD_EEPROM
+
+That will cover 20 defconfigs. Many of the options listed are not suitable as
+they are not related. E.g. it would be odd for CONFIG_CMD_GETTIME to imply
+CMD_EEPROM.
+
+Using this search you can reduce the size of moveconfig patches.
+
+You can automatically add 'imply' statements in the Kconfig with the -a
+option::
+
+    ./tools/moveconfig.py -s -i CONFIG_SCSI \
+            -a CONFIG_ARCH_LS1021A,CONFIG_ARCH_LS1043A
+
+This will add 'imply SCSI' to the two CONFIG options mentioned, assuming that
+the database indicates that they do actually imply CONFIG_SCSI and do not
+already have an 'imply SCSI'.
+
+The output shows where the imply is added::
+
+   18 : CONFIG_ARCH_LS1021A       arch/arm/cpu/armv7/ls102xa/Kconfig:1
+   13 : CONFIG_ARCH_LS1043A       arch/arm/cpu/armv8/fsl-layerscape/Kconfig:11
+   12 : CONFIG_ARCH_LS1046A       arch/arm/cpu/armv8/fsl-layerscape/Kconfig:31
+
+The first number is the number of boards which can avoid having a special
+CONFIG_SCSI option in their defconfig file if this 'imply' is added.
+The location at the right is the Kconfig file and line number where the config
+appears. For example, adding 'imply CONFIG_SCSI' to the 'config ARCH_LS1021A'
+in arch/arm/cpu/armv7/ls102xa/Kconfig at line 1 will help 18 boards to reduce
+the size of their defconfig files.
+
+If you want to add an 'imply' to every imply config in the list, you can use::
+
+    ./tools/moveconfig.py -s -i CONFIG_SCSI -a all
+
+To control which ones are displayed, use -I <list> where list is a list of
+options (use '-I help' to see possible options and their meaning).
+
+To skip showing you options that already have an 'imply' attached, use -A.
+
+When you have finished adding 'imply' options you can regenerate the
+defconfig files for affected boards with something like::
+
+    git show --stat | ./tools/moveconfig.py -s -d -
+
+This will regenerate only those defconfigs changed in the current commit.
+If you start with (say) 100 defconfigs being changed in the commit, and add
+a few 'imply' options as above, then regenerate, hopefully you can reduce the
+number of defconfigs changed in the commit.
+
+
+Available options
+-----------------
+
+ -c, --color
+   Surround each portion of the log with escape sequences to display it
+   in color on the terminal.
+
+ -C, --commit
+   Create a git commit with the changes when the operation is complete. A
+   standard commit message is used which may need to be edited.
+
+ -d, --defconfigs
+  Specify a file containing a list of defconfigs to move.  The defconfig
+  files can be given with shell-style wildcards. Use '-' to read from stdin.
+
+ -n, --dry-run
+   Perform a trial run that does not make any changes.  It is useful to
+   see what is going to happen before one actually runs it.
+
+ -e, --exit-on-error
+   Exit immediately if Make exits with a non-zero status while processing
+   a defconfig file.
+
+ -s, --force-sync
+   Do "make savedefconfig" forcibly for all the defconfig files.
+   If not specified, "make savedefconfig" only occurs for cases
+   where at least one CONFIG was moved.
+
+ -S, --spl
+   Look for moved config options in spl/include/autoconf.mk instead of
+   include/autoconf.mk.  This is useful for moving options for SPL build
+   because SPL related options (mostly prefixed with CONFIG_SPL\_) are
+   sometimes blocked by CONFIG_SPL_BUILD ifdef conditionals.
+
+ -H, --headers-only
+   Only cleanup the headers; skip the defconfig processing
+
+ -j, --jobs
+   Specify the number of threads to run simultaneously.  If not specified,
+   the number of threads is the same as the number of CPU cores.
+
+ -r, --git-ref
+   Specify the git ref to clone for building the autoconf.mk. If unspecified
+   use the CWD. This is useful for when changes to the Kconfig affect the
+   default values and you want to capture the state of the defconfig from
+   before that change was in effect. If in doubt, specify a ref pre-Kconfig
+   changes (use HEAD if Kconfig changes are not committed). Worst case it will
+   take a bit longer to run, but will always do the right thing.
+
+ -v, --verbose
+   Show any build errors as boards are built
+
+ -y, --yes
+   Instead of prompting, automatically go ahead with all operations. This
+   includes cleaning up headers, CONFIG_SYS_EXTRA_OPTIONS, the config whitelist
+   and the README.
+
+To see the complete list of supported options, run::
+
+  tools/moveconfig.py -h
diff --git a/tools/moveconfig.py b/tools/moveconfig.py
index 41dd803c4ef..f3fd75504f7 100755
--- a/tools/moveconfig.py
+++ b/tools/moveconfig.py
@@ -7,296 +7,7 @@
 """
 Move config options from headers to defconfig files.
 
-Since Kconfig was introduced to U-Boot, we have worked on moving
-config options from headers to Kconfig (defconfig).
-
-This tool intends to help this tremendous work.
-
-Installing
-----------
-
-You may need to install 'python3-asteval' for the 'asteval' module.
-
-Usage
------
-
-First, you must edit the Kconfig to add the menu entries for the configs
-you are moving.
-
-And then run this tool giving CONFIG names you want to move.
-For example, if you want to move CONFIG_CMD_USB and CONFIG_SYS_TEXT_BASE,
-simply type as follows:
-
-  $ tools/moveconfig.py CONFIG_CMD_USB CONFIG_SYS_TEXT_BASE
-
-The tool walks through all the defconfig files and move the given CONFIGs.
-
-The log is also displayed on the terminal.
-
-The log is printed for each defconfig as follows:
-
-<defconfig_name>
-    <action1>
-    <action2>
-    <action3>
-    ...
-
-<defconfig_name> is the name of the defconfig.
-
-<action*> shows what the tool did for that defconfig.
-It looks like one of the following:
-
- - Move 'CONFIG_... '
-   This config option was moved to the defconfig
-
- - CONFIG_... is not defined in Kconfig.  Do nothing.
-   The entry for this CONFIG was not found in Kconfig.  The option is not
-   defined in the config header, either.  So, this case can be just skipped.
-
- - CONFIG_... is not defined in Kconfig (suspicious).  Do nothing.
-   This option is defined in the config header, but its entry was not found
-   in Kconfig.
-   There are two common cases:
-     - You forgot to create an entry for the CONFIG before running
-       this tool, or made a typo in a CONFIG passed to this tool.
-     - The entry was hidden due to unmet 'depends on'.
-   The tool does not know if the result is reasonable, so please check it
-   manually.
-
- - 'CONFIG_...' is the same as the define in Kconfig.  Do nothing.
-   The define in the config header matched the one in Kconfig.
-   We do not need to touch it.
-
- - Compiler is missing.  Do nothing.
-   The compiler specified for this architecture was not found
-   in your PATH environment.
-   (If -e option is passed, the tool exits immediately.)
-
- - Failed to process.
-   An error occurred during processing this defconfig.  Skipped.
-   (If -e option is passed, the tool exits immediately on error.)
-
-Finally, you will be asked, Clean up headers? [y/n]:
-
-If you say 'y' here, the unnecessary config defines are removed
-from the config headers (include/configs/*.h).
-It just uses the regex method, so you should not rely on it.
-Just in case, please do 'git diff' to see what happened.
-
-
-How does it work?
------------------
-
-This tool runs configuration and builds include/autoconf.mk for every
-defconfig.  The config options defined in Kconfig appear in the .config
-file (unless they are hidden because of unmet dependency.)
-On the other hand, the config options defined by board headers are seen
-in include/autoconf.mk.  The tool looks for the specified options in both
-of them to decide the appropriate action for the options.  If the given
-config option is found in the .config, but its value does not match the
-one from the board header, the config option in the .config is replaced
-with the define in the board header.  Then, the .config is synced by
-"make savedefconfig" and the defconfig is updated with it.
-
-For faster processing, this tool handles multi-threading.  It creates
-separate build directories where the out-of-tree build is run.  The
-temporary build directories are automatically created and deleted as
-needed.  The number of threads are chosen based on the number of the CPU
-cores of your system although you can change it via -j (--jobs) option.
-
-
-Toolchains
-----------
-
-Appropriate toolchain are necessary to generate include/autoconf.mk
-for all the architectures supported by U-Boot.  Most of them are available
-at the kernel.org site, some are not provided by kernel.org. This tool uses
-the same tools as buildman, so see that tool for setup (e.g. --fetch-arch).
-
-
-Tips and trips
---------------
-
-To sync only X86 defconfigs:
-
-   ./tools/moveconfig.py -s -d <(grep -l X86 configs/*)
-
-or:
-
-   grep -l X86 configs/* | ./tools/moveconfig.py -s -d -
-
-To process CONFIG_CMD_FPGAD only for a subset of configs based on path match:
-
-   ls configs/{hrcon*,iocon*,strider*} | \
-       ./tools/moveconfig.py -Cy CONFIG_CMD_FPGAD -d -
-
-
-Finding implied CONFIGs
------------------------
-
-Some CONFIG options can be implied by others and this can help to reduce
-the size of the defconfig files. For example, CONFIG_X86 implies
-CONFIG_CMD_IRQ, so we can put 'imply CMD_IRQ' under 'config X86' and
-all x86 boards will have that option, avoiding adding CONFIG_CMD_IRQ to
-each of the x86 defconfig files.
-
-This tool can help find such configs. To use it, first build a database:
-
-    ./tools/moveconfig.py -b
-
-Then try to query it:
-
-    ./tools/moveconfig.py -i CONFIG_CMD_IRQ
-    CONFIG_CMD_IRQ found in 311/2384 defconfigs
-    44 : CONFIG_SYS_FSL_ERRATUM_IFC_A002769
-    41 : CONFIG_SYS_FSL_ERRATUM_A007075
-    31 : CONFIG_SYS_FSL_DDR_VER_44
-    28 : CONFIG_ARCH_P1010
-    28 : CONFIG_SYS_FSL_ERRATUM_P1010_A003549
-    28 : CONFIG_SYS_FSL_ERRATUM_SEC_A003571
-    28 : CONFIG_SYS_FSL_ERRATUM_IFC_A003399
-    25 : CONFIG_SYS_FSL_ERRATUM_A008044
-    22 : CONFIG_ARCH_P1020
-    21 : CONFIG_SYS_FSL_DDR_VER_46
-    20 : CONFIG_MAX_PIRQ_LINKS
-    20 : CONFIG_HPET_ADDRESS
-    20 : CONFIG_X86
-    20 : CONFIG_PCIE_ECAM_SIZE
-    20 : CONFIG_IRQ_SLOT_COUNT
-    20 : CONFIG_I8259_PIC
-    20 : CONFIG_CPU_ADDR_BITS
-    20 : CONFIG_RAMBASE
-    20 : CONFIG_SYS_FSL_ERRATUM_A005871
-    20 : CONFIG_PCIE_ECAM_BASE
-    20 : CONFIG_X86_TSC_TIMER
-    20 : CONFIG_I8254_TIMER
-    20 : CONFIG_CMD_GETTIME
-    19 : CONFIG_SYS_FSL_ERRATUM_A005812
-    18 : CONFIG_X86_RUN_32BIT
-    17 : CONFIG_CMD_CHIP_CONFIG
-    ...
-
-This shows a list of config options which might imply CONFIG_CMD_EEPROM along
-with how many defconfigs they cover. From this you can see that CONFIG_X86
-implies CONFIG_CMD_EEPROM. Therefore, instead of adding CONFIG_CMD_EEPROM to
-the defconfig of every x86 board, you could add a single imply line to the
-Kconfig file:
-
-    config X86
-        bool "x86 architecture"
-        ...
-        imply CMD_EEPROM
-
-That will cover 20 defconfigs. Many of the options listed are not suitable as
-they are not related. E.g. it would be odd for CONFIG_CMD_GETTIME to imply
-CMD_EEPROM.
-
-Using this search you can reduce the size of moveconfig patches.
-
-You can automatically add 'imply' statements in the Kconfig with the -a
-option:
-
-    ./tools/moveconfig.py -s -i CONFIG_SCSI \
-            -a CONFIG_ARCH_LS1021A,CONFIG_ARCH_LS1043A
-
-This will add 'imply SCSI' to the two CONFIG options mentioned, assuming that
-the database indicates that they do actually imply CONFIG_SCSI and do not
-already have an 'imply SCSI'.
-
-The output shows where the imply is added:
-
-   18 : CONFIG_ARCH_LS1021A       arch/arm/cpu/armv7/ls102xa/Kconfig:1
-   13 : CONFIG_ARCH_LS1043A       arch/arm/cpu/armv8/fsl-layerscape/Kconfig:11
-   12 : CONFIG_ARCH_LS1046A       arch/arm/cpu/armv8/fsl-layerscape/Kconfig:31
-
-The first number is the number of boards which can avoid having a special
-CONFIG_SCSI option in their defconfig file if this 'imply' is added.
-The location at the right is the Kconfig file and line number where the config
-appears. For example, adding 'imply CONFIG_SCSI' to the 'config ARCH_LS1021A'
-in arch/arm/cpu/armv7/ls102xa/Kconfig at line 1 will help 18 boards to reduce
-the size of their defconfig files.
-
-If you want to add an 'imply' to every imply config in the list, you can use
-
-    ./tools/moveconfig.py -s -i CONFIG_SCSI -a all
-
-To control which ones are displayed, use -I <list> where list is a list of
-options (use '-I help' to see possible options and their meaning).
-
-To skip showing you options that already have an 'imply' attached, use -A.
-
-When you have finished adding 'imply' options you can regenerate the
-defconfig files for affected boards with something like:
-
-    git show --stat | ./tools/moveconfig.py -s -d -
-
-This will regenerate only those defconfigs changed in the current commit.
-If you start with (say) 100 defconfigs being changed in the commit, and add
-a few 'imply' options as above, then regenerate, hopefully you can reduce the
-number of defconfigs changed in the commit.
-
-
-Available options
------------------
-
- -c, --color
-   Surround each portion of the log with escape sequences to display it
-   in color on the terminal.
-
- -C, --commit
-   Create a git commit with the changes when the operation is complete. A
-   standard commit message is used which may need to be edited.
-
- -d, --defconfigs
-  Specify a file containing a list of defconfigs to move.  The defconfig
-  files can be given with shell-style wildcards. Use '-' to read from stdin.
-
- -n, --dry-run
-   Perform a trial run that does not make any changes.  It is useful to
-   see what is going to happen before one actually runs it.
-
- -e, --exit-on-error
-   Exit immediately if Make exits with a non-zero status while processing
-   a defconfig file.
-
- -s, --force-sync
-   Do "make savedefconfig" forcibly for all the defconfig files.
-   If not specified, "make savedefconfig" only occurs for cases
-   where at least one CONFIG was moved.
-
- -S, --spl
-   Look for moved config options in spl/include/autoconf.mk instead of
-   include/autoconf.mk.  This is useful for moving options for SPL build
-   because SPL related options (mostly prefixed with CONFIG_SPL_) are
-   sometimes blocked by CONFIG_SPL_BUILD ifdef conditionals.
-
- -H, --headers-only
-   Only cleanup the headers; skip the defconfig processing
-
- -j, --jobs
-   Specify the number of threads to run simultaneously.  If not specified,
-   the number of threads is the same as the number of CPU cores.
-
- -r, --git-ref
-   Specify the git ref to clone for building the autoconf.mk. If unspecified
-   use the CWD. This is useful for when changes to the Kconfig affect the
-   default values and you want to capture the state of the defconfig from
-   before that change was in effect. If in doubt, specify a ref pre-Kconfig
-   changes (use HEAD if Kconfig changes are not committed). Worst case it will
-   take a bit longer to run, but will always do the right thing.
-
- -v, --verbose
-   Show any build errors as boards are built
-
- -y, --yes
-   Instead of prompting, automatically go ahead with all operations. This
-   includes cleaning up headers, CONFIG_SYS_EXTRA_OPTIONS, the config whitelist
-   and the README.
-
-To see the complete list of supported options, run
-
-  $ tools/moveconfig.py -h
-
+See doc/develop/moveconfig.rst for documentation.
 """
 
 import asteval
-- 
2.32.0.402.g57bb445576-goog

[PATCH 4/5] doc: Fix up outdated moveconfig docs

[Simon Glass]

The examples here are a bit messed up since the command does not match
the documentation. Use a different example instead.

Signed-off-by: Simon Glass <sjg@chromium.org>
---

 doc/develop/moveconfig.rst | 58 +++++++++++++++-----------------------
 1 file changed, 22 insertions(+), 36 deletions(-)

diff --git a/doc/develop/moveconfig.rst b/doc/develop/moveconfig.rst
index aaa155e8c70..dcd4d927e40 100644
--- a/doc/develop/moveconfig.rst
+++ b/doc/develop/moveconfig.rst
@@ -144,50 +144,36 @@ This tool can help find such configs. To use it, first build a database::
 
 Then try to query it::
 
-    ./tools/moveconfig.py -i CONFIG_CMD_IRQ
-    CONFIG_CMD_IRQ found in 311/2384 defconfigs
-    44 : CONFIG_SYS_FSL_ERRATUM_IFC_A002769
-    41 : CONFIG_SYS_FSL_ERRATUM_A007075
-    31 : CONFIG_SYS_FSL_DDR_VER_44
-    28 : CONFIG_ARCH_P1010
-    28 : CONFIG_SYS_FSL_ERRATUM_P1010_A003549
-    28 : CONFIG_SYS_FSL_ERRATUM_SEC_A003571
-    28 : CONFIG_SYS_FSL_ERRATUM_IFC_A003399
-    25 : CONFIG_SYS_FSL_ERRATUM_A008044
-    22 : CONFIG_ARCH_P1020
-    21 : CONFIG_SYS_FSL_DDR_VER_46
-    20 : CONFIG_MAX_PIRQ_LINKS
-    20 : CONFIG_HPET_ADDRESS
-    20 : CONFIG_X86
-    20 : CONFIG_PCIE_ECAM_SIZE
-    20 : CONFIG_IRQ_SLOT_COUNT
-    20 : CONFIG_I8259_PIC
-    20 : CONFIG_CPU_ADDR_BITS
-    20 : CONFIG_RAMBASE
-    20 : CONFIG_SYS_FSL_ERRATUM_A005871
-    20 : CONFIG_PCIE_ECAM_BASE
-    20 : CONFIG_X86_TSC_TIMER
-    20 : CONFIG_I8254_TIMER
-    20 : CONFIG_CMD_GETTIME
-    19 : CONFIG_SYS_FSL_ERRATUM_A005812
-    18 : CONFIG_X86_RUN_32BIT
-    17 : CONFIG_CMD_CHIP_CONFIG
-    ...
-
-This shows a list of config options which might imply CONFIG_CMD_EEPROM along
+   ./tools/moveconfig.py -i CONFIG_I8042_KEYB
+   CONFIG_I8042_KEYB found in 33/5155 defconfigs
+   28 : CONFIG_X86
+   28 : CONFIG_SA_PCIEX_LENGTH
+   28 : CONFIG_HPET_ADDRESS
+   28 : CONFIG_MAX_PIRQ_LINKS
+   28 : CONFIG_I8254_TIMER
+   28 : CONFIG_I8259_PIC
+   28 : CONFIG_RAMBASE
+   28 : CONFIG_IRQ_SLOT_COUNT
+   28 : CONFIG_PCIE_ECAM_SIZE
+   28 : CONFIG_APIC
+   ...
+
+This shows a list of config options which might imply CONFIG_I8042_KEYB along
 with how many defconfigs they cover. From this you can see that CONFIG_X86
-implies CONFIG_CMD_EEPROM. Therefore, instead of adding CONFIG_CMD_EEPROM to
+generally implies CONFIG_I8042_KEYB but not always (28 out of 35). Therefore,
+instead of adding CONFIG_I8042_KEYB to
 the defconfig of every x86 board, you could add a single imply line to the
-Kconfig file:
+Kconfig file::
 
     config X86
         bool "x86 architecture"
         ...
         imply CMD_EEPROM
 
-That will cover 20 defconfigs. Many of the options listed are not suitable as
-they are not related. E.g. it would be odd for CONFIG_CMD_GETTIME to imply
-CMD_EEPROM.
+That will cover 28 defconfigs and you can perhaps find another condition that
+indicates that CONFIG_I8042_KEYB is not needed for the remaining 5 boards. Many
+of the options listed are not suitable as they are not related. E.g. it would be
+odd for CONFIG_RAMBASE to imply CONFIG_I8042_KEYB.
 
 Using this search you can reduce the size of moveconfig patches.
 
-- 
2.32.0.402.g57bb445576-goog

[PATCH 5/5] moveconfig: Update to newer kconfiglib

[Simon Glass]

Some of the more advanced features of this tool don't work anymore since
kconfiglib was update. Update the code accordingly.

Signed-off-by: Simon Glass <sjg@chromium.org>
---

 tools/moveconfig.py | 24 ++++++++++++------------
 1 file changed, 12 insertions(+), 12 deletions(-)

diff --git a/tools/moveconfig.py b/tools/moveconfig.py
index f3fd75504f7..373b395fda4 100755
--- a/tools/moveconfig.py
+++ b/tools/moveconfig.py
@@ -1262,8 +1262,8 @@ def find_kconfig_rules(kconf, config, imply_config):
     """
     sym = kconf.syms.get(imply_config)
     if sym:
-        for sel in sym.get_selected_symbols() | sym.get_implied_symbols():
-            if sel.get_name() == config:
+        for sel, cond in (sym.selects + sym.implies):
+            if sel == config:
                 return sym
     return None
 
@@ -1288,10 +1288,10 @@ def check_imply_rule(kconf, config, imply_config):
     sym = kconf.syms.get(imply_config)
     if not sym:
         return 'cannot find sym'
-    locs = sym.get_def_locations()
-    if len(locs) != 1:
-        return '%d locations' % len(locs)
-    fname, linenum = locs[0]
+    nodes = sym.nodes
+    if len(nodes) != 1:
+        return '%d locations' % len(nodes)
+    fname, linenum = nodes[0].filename, nodes[0].linern
     cwd = os.getcwd()
     if cwd and fname.startswith(cwd):
         fname = fname[len(cwd) + 1:]
@@ -1502,9 +1502,9 @@ def do_imply_config(config_list, add_imply, imply_flags, skip_added,
                                          iconfig[CONFIG_LEN:])
                 kconfig_info = ''
                 if sym:
-                    locs = sym.get_def_locations()
-                    if len(locs) == 1:
-                        fname, linenum = locs[0]
+                    nodes = sym.nodes
+                    if len(nodes) == 1:
+                        fname, linenum = nodes[0].filename, nodes[0].linenr
                         if cwd and fname.startswith(cwd):
                             fname = fname[len(cwd) + 1:]
                         kconfig_info = '%s:%d' % (fname, linenum)
@@ -1514,9 +1514,9 @@ def do_imply_config(config_list, add_imply, imply_flags, skip_added,
                     sym = kconf.syms.get(iconfig[CONFIG_LEN:])
                     fname = ''
                     if sym:
-                        locs = sym.get_def_locations()
-                        if len(locs) == 1:
-                            fname, linenum = locs[0]
+                        nodes = sym.nodes
+                        if len(nodes) == 1:
+                            fname, linenum = nodes[0].filename, nodes[0].linenr
                             if cwd and fname.startswith(cwd):
                                 fname = fname[len(cwd) + 1:]
                     in_arch_board = not sym or (fname.startswith('arch') or
-- 
2.32.0.402.g57bb445576-goog

Re: [PATCH 5/5] moveconfig: Update to newer kconfiglib

[Simon Glass]

Some of the more advanced features of this tool don't work anymore since
kconfiglib was update. Update the code accordingly.

Signed-off-by: Simon Glass <sjg@chromium.org>
---

 tools/moveconfig.py | 24 ++++++++++++------------
 1 file changed, 12 insertions(+), 12 deletions(-)

Applied to u-boot-dm, thanks!

Re: [PATCH 4/5] doc: Fix up outdated moveconfig docs

[Simon Glass]

The examples here are a bit messed up since the command does not match
the documentation. Use a different example instead.

Signed-off-by: Simon Glass <sjg@chromium.org>
---

 doc/develop/moveconfig.rst | 58 +++++++++++++++-----------------------
 1 file changed, 22 insertions(+), 36 deletions(-)

Applied to u-boot-dm, thanks!

Re: [PATCH 3/5] doc: Add docs for the moveconfig tool

[Simon Glass]

Move these docs into htmldocs so they can be read there.

Signed-off-by: Simon Glass <sjg@chromium.org>
---

 doc/develop/index.rst      |   1 +
 doc/develop/moveconfig.rst | 296 +++++++++++++++++++++++++++++++++++++
 tools/moveconfig.py        | 291 +-----------------------------------
 3 files changed, 298 insertions(+), 290 deletions(-)
 create mode 100644 doc/develop/moveconfig.rst

Applied to u-boot-dm, thanks!