[PATCH v4 0/3] docs: Document list of undocumented commands

[PATCH v4 0/3] docs: Document list of undocumented commands

[Hans Ulrich Niedermann]

This patchset adds a script to the buildsystem which determines
the lists of GRUB commands implemented in the source code and
the list of GRUB commands documented in grub.info and documents
the differences.

For that matter, the script will autogenerate a list of the
undocumented commands to be included in grub.info, so that the
undocumented commands are at least listed there so users have
a point to start researching. If the command is at least
mentioned somewhere withing grub.texi, that will be noted.

The two commands pxe_unload and uppermem which were documented
in grub.info but not implemented in the source code are removed
from grub.texi in separate patches in the patchset.

The long term goal of this is to reduce the number of undocumented
GRUB commands to zero and to keep it there.

For conflict free merging, this patchset requires my gitignore
patchset as it adds a gitignore pattern for the file it generates.

Hans Ulrich Niedermann (3):
  docs: Add grub command documentation checks
  docs: Remove docs for non-existing pxe_unload command
  docs: Remove docs for non-existing uppermem command

 .gitignore             |   1 +
 docs/Makefile.am       |  18 +++-
 docs/check-commands.py | 205 +++++++++++++++++++++++++++++++++++++++++
 docs/grub.texi         |  28 +-----
 4 files changed, 225 insertions(+), 27 deletions(-)
 create mode 100644 docs/check-commands.py

Interdiff:
diff --git a/docs/Makefile.am b/docs/Makefile.am
index 8e8500226..a8009e7a9 100644
--- a/docs/Makefile.am
+++ b/docs/Makefile.am
@@ -6,10 +6,12 @@ grub_TEXINFOS = fdl.texi undocumented-commands.texi
 
 EXTRA_DIST = check-commands.py font_char_metrics.png font_char_metrics.txt
 
+# This has only been tested to work without the Automake
+# info-in-builddir option.
 CLEANFILES = undocumented-commands.texi
 BUILT_SOURCES = update-undoc
 update-undoc:
-	$(PYTHON) check-commands.py > undocumented-commands.texi.tmp
+	$(PYTHON) $(srcdir)/check-commands.py > undocumented-commands.texi.tmp
 	@if test -f $(srcdir)/undocumented-commands.texi && $(CMP) undocumented-commands.texi.tmp $(srcdir)/undocumented-commands.texi; then \
 		echo "Not updating undocumented-commands.texi: is up to date"; \
 		rm -f undocumented-commands.texi.tmp; \
-- 
2.26.2

[PATCH v4 1/3] docs: Add grub command documentation checks

[Hans Ulrich Niedermann]

This adds the docs/check-commands.py script which checks whether
grub commands implemented in the git source tree are documented
in docs/grub.texi and vice versa.

During a standard "make" command, BUILT_SOURCES makes sure that
docs/undocumented-commands.texi will be created (if it does not
exist yet) or updated (if it differs from what the script would
generate). Otherwise, the existing file is left alone.

With docs/undocumented-commands.texi being shipped in dist tarballs
alongside grub.info, building GRUB from a dist tarball will still
work without rebuilding grub.info page unless an actual change has
happened which causes a different docs/undocumented-commands.texi
to be generated.

If you run "make grub.info" in the docs/ subdirectory, the
BUILT_SOURCES trick to create or update undocumented-commands.texi
will not work (this is an Automake limitation). So if you want
to avoid the "all" make target, you need to explictly run
"make update-undoc" first.

The generated docs/undocumented-commands.texi file will document each
otherwise undocumented command by describing that it is undocumented,
and by adding that it is mentioned somewhere else in the info page if
that happens to be the case.

The build time requirements of the make target's build rule are
already required by other parts of the GRUB buildsystem:

  * $(CMP)
    compare two files by content

  * $(PYTHON)
    check-commands.py has been written to run on Python 2 and 3
    (tested with Python 2.7 and Python 3.7)

  * texinfo tools
    When building from a git source tree, texinfo is required
    to rebuild docs/grub.info. When building from a dist tarball,
    texinfo is not required. This is standard Automake behaviour
    which we do not change.

    However, when building from a dist tarball if you ever run
    "make clean", then the texinfo toolchain will be required as
    docs/undocumented-commands.texi will be rebuilt and thus
    a rebuild of docs/grub.info will be triggered.

    I do not expect distribution package builds from dist
    tarball to use "make clean", so I do not expect this new
    requirement for the texinfo tools to have much of an impact.

The docs/check-commands.py script runs fast enough to not perceivably
slow down a "make" on the grub source tree. The script basically reads
and more or less greps through docs/grub.texi and all *.c source files,
and that happens very quickly.

Signed-off-by: Hans Ulrich Niedermann <hun@n-dimensional.de>

 create mode 100644 docs/check-commands.py

diff --git a/.gitignore b/.gitignore
index aa180fa89..b7a853b90 100644
--- a/.gitignore
+++ b/.gitignore
@@ -96,6 +96,7 @@ widthspec.bin
 /docs/*.info-[0-9]*
 /docs/stamp-1
 /docs/stamp-vti
+/docs/undocumented-commands.texi
 /docs/version-dev.texi
 /docs/version.texi
 /ehci_test
diff --git a/docs/Makefile.am b/docs/Makefile.am
index 93eb39627..a8009e7a9 100644
--- a/docs/Makefile.am
+++ b/docs/Makefile.am
@@ -2,8 +2,20 @@ AUTOMAKE_OPTIONS = subdir-objects
 
 # AM_MAKEINFOFLAGS = --no-split --no-validate
 info_TEXINFOS = grub.texi grub-dev.texi
-grub_TEXINFOS = fdl.texi
-
-EXTRA_DIST = font_char_metrics.png font_char_metrics.txt
+grub_TEXINFOS = fdl.texi undocumented-commands.texi
 
+EXTRA_DIST = check-commands.py font_char_metrics.png font_char_metrics.txt
 
+# This has only been tested to work without the Automake
+# info-in-builddir option.
+CLEANFILES = undocumented-commands.texi
+BUILT_SOURCES = update-undoc
+update-undoc:
+	$(PYTHON) $(srcdir)/check-commands.py > undocumented-commands.texi.tmp
+	@if test -f $(srcdir)/undocumented-commands.texi && $(CMP) undocumented-commands.texi.tmp $(srcdir)/undocumented-commands.texi; then \
+		echo "Not updating undocumented-commands.texi: is up to date"; \
+		rm -f undocumented-commands.texi.tmp; \
+	else \
+		echo "Updating undocumented-commands.texi"; \
+		mv -f undocumented-commands.texi.tmp $(srcdir)/undocumented-commands.texi; \
+	fi
diff --git a/docs/check-commands.py b/docs/check-commands.py
new file mode 100644
index 000000000..4174b954a
--- /dev/null
+++ b/docs/check-commands.py
@@ -0,0 +1,205 @@
+#!/usr/bin/env python
+#
+# check-commands.py - compare commands defined in grub.texi the *.c sources
+#
+# Usage:
+#     check-commands.py > undocumented-commands.texi.new
+#
+# There are no command line parameters. The locations of grub.texi and
+# the *.c source files are determined relative to the location of the
+# script file. The only output happens on stdout, in texinfo format
+# for inclusione into grub.texi via '@include'.
+
+from __future__ import print_function
+
+import os
+import re
+
+import sys
+
+
+class CommandChecker:
+
+    def __init__(self):
+        srcdir, self.prog = os.path.split(__file__)
+        self.top_srcdir = os.path.dirname(os.path.abspath(srcdir))
+
+    def read_texi_text_commands(self, texi_filename):
+        texi_pathname = os.path.join(self.top_srcdir, 'docs', texi_filename)
+        command_re = re.compile(r'@command\{([a-zA-Z0-9_-]+)\}')
+        commands = set()
+        with open(texi_pathname) as texi:
+            for line in texi.readlines():
+                for m in command_re.finditer(line):
+                    commands.add(m[1])
+        return commands
+
+    def read_texi_command_menu(self, texi_filename):
+        texi_pathname = os.path.join(self.top_srcdir, 'docs', texi_filename)
+        menuentry_re = re.compile(r'\* ([^:]+)::')
+        commands_node_re = re.compile(r'^@node .+ commands$')
+        commands = set()
+
+        # Avoid using enum for the state machine state (for compatibility)
+        waiting_for_node = True
+        waiting_for_menu = False
+        collecting_commands = False
+
+        with open(texi_pathname) as texi:
+            for line in texi.readlines():
+                line = line.strip()
+                if line.startswith('@comment'):
+                    continue
+                if waiting_for_node:
+                    if commands_node_re.match(line):
+                        waiting_for_node = False
+                        waiting_for_menu = True
+                        continue
+                if waiting_for_menu:
+                    if line == '@menu':
+                        waiting_for_menu = False
+                        collecting_commands = True
+                        continue
+                if collecting_commands:
+                    if line == '@end menu':
+                        collecting_commands = False
+                        waiting_for_node = True
+                        continue
+                    m = menuentry_re.match(line)
+                    commands.add(m.group(1))
+        return commands
+
+    def read_src_commands(self):
+        top = os.path.join(self.top_srcdir, 'grub-core')
+        register_re = re.compile(r'grub_register_(command|command_p1|extcmd)\s*\("([a-z0-9A-Z_\[]+)",')
+        commands = set()
+        for dirpath, dirnames, filenames in os.walk(top):
+            for filename in filenames:
+                filepath = os.path.join(dirpath, filename)
+                filepath_ext = os.path.splitext(filepath)[1]
+                if filepath_ext == '.c':
+                    with open(filepath) as cfile:
+                        for line in cfile.readlines():
+                            for m in register_re.finditer(line):
+                                commands.add(m.group(2))
+        return commands
+
+
+def write_undocumented_commands(undocumented_commands,
+                                mentioned_commands):
+    print("""\
+@node Undocumented commands
+@section The list of undocumented commands
+""")
+
+    if not undocumented_commands:
+        print("""
+There appear to be no undocumented commands at this time.
+""")
+        return
+
+    print("""\
+These commands are implemented in the grub software but still need to be
+documented and sorted into categories.
+""")
+
+    sorted_undocumented_commands = sorted(list(undocumented_commands))
+
+    # Fit the longest command into the format string for the menu entries
+    maxlen_str = sorted(list(undocumented_commands),
+                        key=lambda cmd: len(cmd),
+                        reverse=True)[0]
+    fmt = '* %%-%ds %%s' % (2+len(maxlen_str))
+
+    print("@menu")
+    for cmd in sorted_undocumented_commands:
+        if cmd in mentioned_commands:
+            print(fmt % ("%s::" % cmd, "Undocumented command (but mentioned somewhere)"))
+        else:
+            print(fmt % ("%s::" % cmd, "Undocumented command"))
+    print("@end menu")
+    print()
+
+    for cmd in sorted_undocumented_commands:
+        print("@node %s" % cmd)
+        print("@subsection %s" % cmd)
+        print()
+        print("The grub command @command{%s} has not been documented properly yet." % cmd)
+        if cmd in mentioned_commands:
+            print("""
+However, the @command{%s} command @emph{is} mentioned
+somewhere within this info file, so you might still find some
+interesting information there.""" % cmd)
+        print()
+
+
+def print_set(st):
+    for n, item in enumerate(sorted(list(st)), start=1):
+        print("@comment", "  %d." % n, item)
+
+
+def main():
+    cc = CommandChecker()
+
+    print("""\
+@comment Automatically generated by the %s script.
+@comment Do not modify this generated file.
+@comment
+@comment If you want to document some of the commands listed below, feel
+@comment free to copy over some of the @nodes and @subsections below
+@comment into the grub.texi file proper and re-generate this file.
+@comment""" % cc.prog)
+
+    # This gives a few good hints about what commands are at least
+    # mentioned somewhere which can be helpful for finding out more
+    # about commands.
+    texi_text_commands = cc.read_texi_text_commands('grub.texi')
+    print("@comment", "Commands mentioned somewhere in the grub.texi text:")
+    print_set(texi_text_commands)
+
+    texi_node_commands = cc.read_texi_command_menu('grub.texi')
+    # print("@comment", "Commands documented in grub.texi node/subsection:")
+    # print_set(texi_node_commands)
+
+    src_commands = cc.read_src_commands()
+    # print("@comment", "Commands registered in grub source:")
+    # print_set(src_commands)
+
+    node_without_src = texi_node_commands - src_commands
+    if node_without_src:
+        print("@comment",
+              "Cmds documented in grub.texi node/subsection but not registered in source:")
+        print_set(node_without_src)
+        print("@comment")
+    else:
+        print("""\
+@comment There appear to be no commands documented in a grub.texi
+@comment node/subsection but not registered in grub source.
+@comment""")
+
+    src_without_node = src_commands - texi_node_commands
+    if src_without_node:
+        print("@comment",
+              "Cmds registered in source but not documented in grub.texi node/subsection:")
+        print_set(src_without_node)
+    else:
+        print("""\
+@comment All commands registered in the source appear to have been
+@comment documented in a grub.texi node/subsection. Congratulations!
+@comment""")
+
+    print()
+
+    write_undocumented_commands(src_without_node, texi_text_commands)
+
+    # Once grub.texi actually documents all commands, we can uncomment
+    # this and actually fail if the set of implemented commands and
+    # the set of documented commands differ in any way.
+    # if ((len(node_without_src) > 0) or (len(src_without_node) > 0)):
+    #     sys.exit(1)
+
+    sys.exit(0)
+
+
+if __name__ == '__main__':
+    main()
diff --git a/docs/grub.texi b/docs/grub.texi
index d6408d242..0865ffa17 100644
--- a/docs/grub.texi
+++ b/docs/grub.texi
@@ -3764,6 +3764,7 @@ shell}.
 * General commands::
 * Command-line and menu entry commands::
 * Networking commands::
+* Undocumented commands::
 @end menu
 
 
@@ -5636,6 +5637,9 @@ is given, use default list of servers.
 @end deffn
 
 
+@include undocumented-commands.texi
+
+
 @node Internationalisation
 @chapter Internationalisation
 
-- 
2.26.2

[PATCH v4 2/3] docs: Remove docs for non-existing pxe_unload command

[Hans Ulrich Niedermann]

Remove the documentation of the pxe_unload command from the
docs/grub.texi file.

The pxe_unload command is not implemented in the grub source
at this time at all. It appears to have been removed in commit
71a78acb0648d3d73c95ab0f021f907499aacc0 (from 2011-07-05).

Signed-off-by: Hans Ulrich Niedermann <hun@n-dimensional.de>

diff --git a/docs/grub.texi b/docs/grub.texi
index 0865ffa17..0f7886bb6 100644
--- a/docs/grub.texi
+++ b/docs/grub.texi
@@ -4012,7 +4012,6 @@ you forget a command, you can run the command @command{help}
 * password_pbkdf2::             Set a hashed password
 * play::                        Play a tune
 * probe::                       Retrieve device info
-* pxe_unload::                  Unload the PXE environment
 * rdmsr::                       Read values from model-specific registers
 * read::                        Read user input
 * reboot::                      Reboot your computer
@@ -4863,16 +4862,6 @@ The option @option{--part-uuid} is currently only implemented for GPT-formatted
 @end deffn
 
 
-@node pxe_unload
-@subsection pxe_unload
-
-@deffn Command pxe_unload
-Unload the PXE environment (@pxref{Network}).
-
-This command is only available on PC BIOS systems.
-@end deffn
-
-
 @node rdmsr
 @subsection rdmsr
 
-- 
2.26.2

[PATCH v4 3/3] docs: Remove docs for non-existing uppermem command

[Hans Ulrich Niedermann]

Remove all documentation of and mentions of the uppermem
command from the docs/grub.texi file.

The uppermem command is not implemented in the GRUB source
at all and appears to never have been implemented despite
former plans to add an uppermem command.

To reduce user confusion, this even removes the paragraph
describing how GRUB's uppermem command was supposed to
complement the Linux kernel's mem= parameter.

Signed-off-by: Hans Ulrich Niedermann <hun@n-dimensional.de>

diff --git a/docs/grub.texi b/docs/grub.texi
index 0f7886bb6..d51ad0e25 100644
--- a/docs/grub.texi
+++ b/docs/grub.texi
@@ -1114,12 +1114,6 @@ grub> @kbd{initrd16 /initrd}
 Finally, run the command @command{boot} (@pxref{boot}).
 @end enumerate
 
-@strong{Caution:} If you use an initrd and specify the @samp{mem=}
-option to the kernel to let it use less than actual memory size, you
-will also have to specify the same memory size to GRUB. To let GRUB know
-the size, run the command @command{uppermem} @emph{before} loading the
-kernel. @xref{uppermem}, for more information.
-
 
 @node NetBSD
 @subsection NetBSD
@@ -4031,7 +4025,6 @@ you forget a command, you can run the command @command{help}
 * true::                        Do nothing, successfully
 * trust::                       Add public key to list of trusted keys
 * unset::                       Unset an environment variable
-* uppermem::                    Set the upper memory size
 @comment * vbeinfo::                     List available video modes
 * verify_detached::             Verify detached digital signature
 * videoinfo::                   List available video modes
@@ -5349,12 +5342,6 @@ Unset the environment variable @var{envvar}.
 @end deffn
 
 
-@node uppermem
-@subsection uppermem
-
-This command is not yet implemented for GRUB 2, although it is planned.
-
-
 @ignore
 @node vbeinfo
 @subsection vbeinfo
-- 
2.26.2

Re: [PATCH v4 2/3] docs: Remove docs for non-existing pxe_unload command

[Daniel Kiper]

On Mon, May 11, 2020 at 01:12:30AM +0200, Hans Ulrich Niedermann wrote:
> Remove the documentation of the pxe_unload command from the
> docs/grub.texi file.
>
> The pxe_unload command is not implemented in the grub source
> at this time at all. It appears to have been removed in commit
> 71a78acb0648d3d73c95ab0f021f907499aacc0 (from 2011-07-05).

It seems to me you though about commit 671a78acb (cleanup pxe and efi
network release). I will fix it before push...

> Signed-off-by: Hans Ulrich Niedermann <hun@n-dimensional.de>

Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>

Daniel

Re: [PATCH v4 3/3] docs: Remove docs for non-existing uppermem command

[Daniel Kiper]

On Mon, May 11, 2020 at 01:12:31AM +0200, Hans Ulrich Niedermann wrote:
> Remove all documentation of and mentions of the uppermem
> command from the docs/grub.texi file.
>
> The uppermem command is not implemented in the GRUB source
> at all and appears to never have been implemented despite
> former plans to add an uppermem command.
>
> To reduce user confusion, this even removes the paragraph
> describing how GRUB's uppermem command was supposed to
> complement the Linux kernel's mem= parameter.
>
> Signed-off-by: Hans Ulrich Niedermann <hun@n-dimensional.de>

Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>

Daniel

Re: [PATCH v4 1/3] docs: Add grub command documentation checks

[Alex Burmashev]

Reviewed-by: Alex Burmashev <alexander.burmashev@oracle.com>

11-May-20 1:12 AM, Hans Ulrich Niedermann пишет:
> This adds the docs/check-commands.py script which checks whether
> grub commands implemented in the git source tree are documented
> in docs/grub.texi and vice versa.
>
> During a standard "make" command, BUILT_SOURCES makes sure that
> docs/undocumented-commands.texi will be created (if it does not
> exist yet) or updated (if it differs from what the script would
> generate). Otherwise, the existing file is left alone.
>
> With docs/undocumented-commands.texi being shipped in dist tarballs
> alongside grub.info, building GRUB from a dist tarball will still
> work without rebuilding grub.info page unless an actual change has
> happened which causes a different docs/undocumented-commands.texi
> to be generated.
>
> If you run "make grub.info" in the docs/ subdirectory, the
> BUILT_SOURCES trick to create or update undocumented-commands.texi
> will not work (this is an Automake limitation). So if you want
> to avoid the "all" make target, you need to explictly run
> "make update-undoc" first.
>
> The generated docs/undocumented-commands.texi file will document each
> otherwise undocumented command by describing that it is undocumented,
> and by adding that it is mentioned somewhere else in the info page if
> that happens to be the case.
>
> The build time requirements of the make target's build rule are
> already required by other parts of the GRUB buildsystem:
>
>   * $(CMP)
>     compare two files by content
>
>   * $(PYTHON)
>     check-commands.py has been written to run on Python 2 and 3
>     (tested with Python 2.7 and Python 3.7)
>
>   * texinfo tools
>     When building from a git source tree, texinfo is required
>     to rebuild docs/grub.info. When building from a dist tarball,
>     texinfo is not required. This is standard Automake behaviour
>     which we do not change.
>
>     However, when building from a dist tarball if you ever run
>     "make clean", then the texinfo toolchain will be required as
>     docs/undocumented-commands.texi will be rebuilt and thus
>     a rebuild of docs/grub.info will be triggered.
>
>     I do not expect distribution package builds from dist
>     tarball to use "make clean", so I do not expect this new
>     requirement for the texinfo tools to have much of an impact.
>
> The docs/check-commands.py script runs fast enough to not perceivably
> slow down a "make" on the grub source tree. The script basically reads
> and more or less greps through docs/grub.texi and all *.c source files,
> and that happens very quickly.
>
> Signed-off-by: Hans Ulrich Niedermann <hun@n-dimensional.de>
>
>  create mode 100644 docs/check-commands.py
>
> diff --git a/.gitignore b/.gitignore
> index aa180fa89..b7a853b90 100644
> --- a/.gitignore
> +++ b/.gitignore
> @@ -96,6 +96,7 @@ widthspec.bin
>  /docs/*.info-[0-9]*
>  /docs/stamp-1
>  /docs/stamp-vti
> +/docs/undocumented-commands.texi
>  /docs/version-dev.texi
>  /docs/version.texi
>  /ehci_test
> diff --git a/docs/Makefile.am b/docs/Makefile.am
> index 93eb39627..a8009e7a9 100644
> --- a/docs/Makefile.am
> +++ b/docs/Makefile.am
> @@ -2,8 +2,20 @@ AUTOMAKE_OPTIONS = subdir-objects
>  
>  # AM_MAKEINFOFLAGS = --no-split --no-validate
>  info_TEXINFOS = grub.texi grub-dev.texi
> -grub_TEXINFOS = fdl.texi
> -
> -EXTRA_DIST = font_char_metrics.png font_char_metrics.txt
> +grub_TEXINFOS = fdl.texi undocumented-commands.texi
>  
> +EXTRA_DIST = check-commands.py font_char_metrics.png font_char_metrics.txt
>  
> +# This has only been tested to work without the Automake
> +# info-in-builddir option.
> +CLEANFILES = undocumented-commands.texi
> +BUILT_SOURCES = update-undoc
> +update-undoc:
> +	$(PYTHON) $(srcdir)/check-commands.py > undocumented-commands.texi.tmp
> +	@if test -f $(srcdir)/undocumented-commands.texi && $(CMP) undocumented-commands.texi.tmp $(srcdir)/undocumented-commands.texi; then \
> +		echo "Not updating undocumented-commands.texi: is up to date"; \
> +		rm -f undocumented-commands.texi.tmp; \
> +	else \
> +		echo "Updating undocumented-commands.texi"; \
> +		mv -f undocumented-commands.texi.tmp $(srcdir)/undocumented-commands.texi; \
> +	fi
> diff --git a/docs/check-commands.py b/docs/check-commands.py
> new file mode 100644
> index 000000000..4174b954a
> --- /dev/null
> +++ b/docs/check-commands.py
> @@ -0,0 +1,205 @@
> +#!/usr/bin/env python
> +#
> +# check-commands.py - compare commands defined in grub.texi the *.c sources
> +#
> +# Usage:
> +#     check-commands.py > undocumented-commands.texi.new
> +#
> +# There are no command line parameters. The locations of grub.texi and
> +# the *.c source files are determined relative to the location of the
> +# script file. The only output happens on stdout, in texinfo format
> +# for inclusione into grub.texi via '@include'.
> +
> +from __future__ import print_function
> +
> +import os
> +import re
> +
> +import sys
> +
> +
> +class CommandChecker:
> +
> +    def __init__(self):
> +        srcdir, self.prog = os.path.split(__file__)
> +        self.top_srcdir = os.path.dirname(os.path.abspath(srcdir))
> +
> +    def read_texi_text_commands(self, texi_filename):
> +        texi_pathname = os.path.join(self.top_srcdir, 'docs', texi_filename)
> +        command_re = re.compile(r'@command\{([a-zA-Z0-9_-]+)\}')
> +        commands = set()
> +        with open(texi_pathname) as texi:
> +            for line in texi.readlines():
> +                for m in command_re.finditer(line):
> +                    commands.add(m[1])
> +        return commands
> +
> +    def read_texi_command_menu(self, texi_filename):
> +        texi_pathname = os.path.join(self.top_srcdir, 'docs', texi_filename)
> +        menuentry_re = re.compile(r'\* ([^:]+)::')
> +        commands_node_re = re.compile(r'^@node .+ commands$')
> +        commands = set()
> +
> +        # Avoid using enum for the state machine state (for compatibility)
> +        waiting_for_node = True
> +        waiting_for_menu = False
> +        collecting_commands = False
> +
> +        with open(texi_pathname) as texi:
> +            for line in texi.readlines():
> +                line = line.strip()
> +                if line.startswith('@comment'):
> +                    continue
> +                if waiting_for_node:
> +                    if commands_node_re.match(line):
> +                        waiting_for_node = False
> +                        waiting_for_menu = True
> +                        continue
> +                if waiting_for_menu:
> +                    if line == '@menu':
> +                        waiting_for_menu = False
> +                        collecting_commands = True
> +                        continue
> +                if collecting_commands:
> +                    if line == '@end menu':
> +                        collecting_commands = False
> +                        waiting_for_node = True
> +                        continue
> +                    m = menuentry_re.match(line)
> +                    commands.add(m.group(1))
> +        return commands
> +
> +    def read_src_commands(self):
> +        top = os.path.join(self.top_srcdir, 'grub-core')
> +        register_re = re.compile(r'grub_register_(command|command_p1|extcmd)\s*\("([a-z0-9A-Z_\[]+)",')
> +        commands = set()
> +        for dirpath, dirnames, filenames in os.walk(top):
> +            for filename in filenames:
> +                filepath = os.path.join(dirpath, filename)
> +                filepath_ext = os.path.splitext(filepath)[1]
> +                if filepath_ext == '.c':
> +                    with open(filepath) as cfile:
> +                        for line in cfile.readlines():
> +                            for m in register_re.finditer(line):
> +                                commands.add(m.group(2))
> +        return commands
> +
> +
> +def write_undocumented_commands(undocumented_commands,
> +                                mentioned_commands):
> +    print("""\
> +@node Undocumented commands
> +@section The list of undocumented commands
> +""")
> +
> +    if not undocumented_commands:
> +        print("""
> +There appear to be no undocumented commands at this time.
> +""")
> +        return
> +
> +    print("""\
> +These commands are implemented in the grub software but still need to be
> +documented and sorted into categories.
> +""")
> +
> +    sorted_undocumented_commands = sorted(list(undocumented_commands))
> +
> +    # Fit the longest command into the format string for the menu entries
> +    maxlen_str = sorted(list(undocumented_commands),
> +                        key=lambda cmd: len(cmd),
> +                        reverse=True)[0]
> +    fmt = '* %%-%ds %%s' % (2+len(maxlen_str))
> +
> +    print("@menu")
> +    for cmd in sorted_undocumented_commands:
> +        if cmd in mentioned_commands:
> +            print(fmt % ("%s::" % cmd, "Undocumented command (but mentioned somewhere)"))
> +        else:
> +            print(fmt % ("%s::" % cmd, "Undocumented command"))
> +    print("@end menu")
> +    print()
> +
> +    for cmd in sorted_undocumented_commands:
> +        print("@node %s" % cmd)
> +        print("@subsection %s" % cmd)
> +        print()
> +        print("The grub command @command{%s} has not been documented properly yet." % cmd)
> +        if cmd in mentioned_commands:
> +            print("""
> +However, the @command{%s} command @emph{is} mentioned
> +somewhere within this info file, so you might still find some
> +interesting information there.""" % cmd)
> +        print()
> +
> +
> +def print_set(st):
> +    for n, item in enumerate(sorted(list(st)), start=1):
> +        print("@comment", "  %d." % n, item)
> +
> +
> +def main():
> +    cc = CommandChecker()
> +
> +    print("""\
> +@comment Automatically generated by the %s script.
> +@comment Do not modify this generated file.
> +@comment
> +@comment If you want to document some of the commands listed below, feel
> +@comment free to copy over some of the @nodes and @subsections below
> +@comment into the grub.texi file proper and re-generate this file.
> +@comment""" % cc.prog)
> +
> +    # This gives a few good hints about what commands are at least
> +    # mentioned somewhere which can be helpful for finding out more
> +    # about commands.
> +    texi_text_commands = cc.read_texi_text_commands('grub.texi')
> +    print("@comment", "Commands mentioned somewhere in the grub.texi text:")
> +    print_set(texi_text_commands)
> +
> +    texi_node_commands = cc.read_texi_command_menu('grub.texi')
> +    # print("@comment", "Commands documented in grub.texi node/subsection:")
> +    # print_set(texi_node_commands)
> +
> +    src_commands = cc.read_src_commands()
> +    # print("@comment", "Commands registered in grub source:")
> +    # print_set(src_commands)
> +
> +    node_without_src = texi_node_commands - src_commands
> +    if node_without_src:
> +        print("@comment",
> +              "Cmds documented in grub.texi node/subsection but not registered in source:")
> +        print_set(node_without_src)
> +        print("@comment")
> +    else:
> +        print("""\
> +@comment There appear to be no commands documented in a grub.texi
> +@comment node/subsection but not registered in grub source.
> +@comment""")
> +
> +    src_without_node = src_commands - texi_node_commands
> +    if src_without_node:
> +        print("@comment",
> +              "Cmds registered in source but not documented in grub.texi node/subsection:")
> +        print_set(src_without_node)
> +    else:
> +        print("""\
> +@comment All commands registered in the source appear to have been
> +@comment documented in a grub.texi node/subsection. Congratulations!
> +@comment""")
> +
> +    print()
> +
> +    write_undocumented_commands(src_without_node, texi_text_commands)
> +
> +    # Once grub.texi actually documents all commands, we can uncomment
> +    # this and actually fail if the set of implemented commands and
> +    # the set of documented commands differ in any way.
> +    # if ((len(node_without_src) > 0) or (len(src_without_node) > 0)):
> +    #     sys.exit(1)
> +
> +    sys.exit(0)
> +
> +
> +if __name__ == '__main__':
> +    main()
> diff --git a/docs/grub.texi b/docs/grub.texi
> index d6408d242..0865ffa17 100644
> --- a/docs/grub.texi
> +++ b/docs/grub.texi
> @@ -3764,6 +3764,7 @@ shell}.
>  * General commands::
>  * Command-line and menu entry commands::
>  * Networking commands::
> +* Undocumented commands::
>  @end menu
>  
>  
> @@ -5636,6 +5637,9 @@ is given, use default list of servers.
>  @end deffn
>  
>  
> +@include undocumented-commands.texi
> +
> +
>  @node Internationalisation
>  @chapter Internationalisation
>