* 119 grub commands not documented in grub.texi
@ 2020-04-18 10:53 Hans Ulrich Niedermann
2020-04-22 10:10 ` Daniel Kiper
0 siblings, 1 reply; 9+ messages in thread
From: Hans Ulrich Niedermann @ 2020-04-18 10:53 UTC (permalink / raw)
To: grub-devel
Hi,
I have noticed that there are two commands documented in grub.texi
which appear not to occur anywhere within the grub source code:
'pxe_unload' and 'uppermem'.
@node pxe_unload
@subsection pxe_unload
@deffn Command pxe_unload
Unload the PXE environment (@pxref{Network}).
This command is only available on PC BIOS systems.
@end deffn
However, judging from the source code ("git grep pxe_unload"),
pxe_unload is not available at all, so the documentation is wrong when
it claims that the command were available on PC BIOS systems.
Should 'pxe_unload' be removed from grub.texi, or does it need a notice
that there are plans to implement it?
Regarding 'uppermem', there is only
@node uppermem
@subsection uppermem
This command is not yet implemented for GRUB 2, although it is planned.
This may be a valid state for the "uppermem" command.
So far for those two commands.
Then, a bit more surprisingly, there appear to be 119 (yes, more than
100) grub commands which are registered within the grub source code but
are not documented within grub.texi.
* When do I count a command as documented in grub.texi?
Every @node/@subsection in a "@section * commands" of
"@chapter Commands".
* What do I count as a command registered within grub source?
Whatever string appears as the first argument in a call to any of
the following functions:
* grub_register_command
* grub_register_command_p1
* grub_register_extcmd
Some of these undocumented commands look like they could be relatively
easily be grouped into a @section and described there (e.g. the 'break',
'continue', 'exit', 'return' commands for a group of shell like
commands, maybe together with '[').
I am absolutely certain that I will not find the time to write useful
documentation for a significant portion of these undocumented commands
within the grub 2.06 release timeframe (i.e. until June 2020), or even
just determine which ones I think actually need to or should be
documented publically.
So if someone wants their favourite commands documented for 2.06, they
will need to write that documentation themselves. I will just try to
add tentative documentation for @command{module2} and
@command{multiboot2} so that booting of a multiboot2 kernel will at
least be mentioned in grub.texi at all.
If you want me to, I can submit a patch for the check-commands.py
script to autogenerate a list of undocumented commands which should be
documented which can then be auto-included in grub.texi and appear in
the generated 'grub.info'. This would give a reader of the grub manual
at least the knowledge that those command actually exist, and could be
coupled with a call for help documenting those commands right inside
grub.texi itself.
Uli
####################################################################
check-commands.py - compare documented and implemented commands
Commands documented in a grub.texi menu but not registered in grub source:
1. pxe_unload
2. uppermem
Commands registered in grub source but not documented in a grub.texi node:
1. all_functional_test
2. appleloader
3. backtrace
4. boottime
5. break
6. cacheinfo
7. cbmemc
8. cmosset
9. continue
10. coreboot_boottime
11. cutmem
12. dump
13. efiemu_loadcore
14. efiemu_prepare
15. efiemu_unload
16. exit
17. extract_entries_configfile
18. extract_entries_source
19. extract_legacy_entries_configfile
20. extract_legacy_entries_source
21. extract_syslinux_entries_configfile
22. extract_syslinux_entries_source
23. fakebios
24. file
25. fix_video
26. fpswa
27. freedos
28. functional_test
29. fwsetup
30. gdbstub
31. gdbstub_break
32. gdbstub_stop
33. hdparm
34. hello
35. hexdump
36. hexdump_random
37. inb
38. inl
39. inw
40. jpegtest
41. keymap
42. kfreebsd
43. kfreebsd_loadenv
44. kfreebsd_module
45. kfreebsd_module_elf
46. knetbsd
47. knetbsd_module
48. knetbsd_module_elf
49. kopenbsd
50. kopenbsd_ramdisk
51. legacy_check_password
52. legacy_configfile
53. legacy_initrd
54. legacy_initrd_nounzip
55. legacy_kernel
56. legacy_password
57. legacy_source
58. loadbios
59. lsacpi
60. lsapm
61. lscoreboot
62. lsdev
63. lsefi
64. lsefimmap
65. lsefisystab
66. lsmmap
67. lspci
68. lssal
69. lsspd
70. macppcbless
71. mactelbless
72. module2
73. multiboot2
74. ntldr
75. outb
76. outl
77. outw
78. pcidump
79. plan9
80. pngtest
81. pxechainloader
82. read_byte
83. read_dword
84. read_word
85. return
86. setparams
87. setpci
88. shift
89. suspend
90. syslinux_configfile
91. syslinux_source
92. test_blockarg
93. testload
94. testspeed
95. tgatest
96. time
97. tr
98. truecrypt
99. usb
100. vbeinfo
101. vbetest
102. videotest
103. write_byte
104. write_dword
105. write_word
106. xen_cat
107. xen_ls
108. xnu_devprop_load
109. xnu_kernel
110. xnu_kernel64
111. xnu_kext
112. xnu_kextdir
113. xnu_mkext
114. xnu_ramdisk
115. xnu_resume
116. xnu_splash
117. xnu_uuid
118. zfsinfo
119. zfskey
####################################################################
#!/usr/bin/env python
from __future__ import print_function
import os
import re
import sys
class CommandChecker:
def __init__(self):
srcdir, self.prog = os.path.split(__file__)
self.top_srcdir = os.path.dirname(os.path.abspath(srcdir))
def read_texi_text_commands(self, texi_filename):
texi_pathname = os.path.join(self.top_srcdir, 'docs', texi_filename)
r = re.compile(r'@command\{([a-zA-Z0-9_-]+)\}')
commands = set()
with open(texi_pathname) as texi:
for line in texi.readlines():
for m in r.finditer(line):
commands.add(m[1])
return commands
def read_texi_command_menu(self, texi_filename):
texi_pathname = os.path.join(self.top_srcdir, 'docs', texi_filename)
r = re.compile(r'\* ([^:]+)::')
n = re.compile(r'^@node .+ commands$')
commands = set()
waiting_for_node = True
waiting_for_menu = False
collecting_commands = False
with open(texi_pathname) as texi:
for line in texi.readlines():
line = line.strip()
if line.startswith('@comment'):
continue
if waiting_for_node:
if n.match(line):
# print("@comment", line)
waiting_for_node = False
waiting_for_menu = True
continue
if waiting_for_menu:
if line == '@menu':
waiting_for_menu = False
collecting_commands = True
continue
if collecting_commands:
if line == '@end menu':
collecting_commands = False
waiting_for_node = True
continue
m = r.match(line)
# print("@comment ", m[1])
commands.add(m.group(1))
return commands
def read_src_commands(self):
top = os.path.join(self.top_srcdir, 'grub-core')
r =
re.compile(r'grub_register_(command|command_p1|extcmd)\s*\("([a-z0-9A-Z_\[]+)",')
commands = set() for dirpath, dirnames, filenames in
os.walk(top): for fn in filenames:
fp = os.path.join(dirpath, fn)
fpe = os.path.splitext(fp)[1]
if fpe == '.c':
with open(fp) as cfile:
for line in cfile.readlines():
for m in r.finditer(line):
commands.add(m.group(2))
return commands
def write_undocumented_commands(commands):
print("""\
@node Undocumented commands
@section The list of undocumented commands
These commands still need to be documented and sorted into categories.
""")
maxlen_str = sorted(list(commands), key=lambda cmd: len(cmd),
reverse=True)[0]
fmt = '* %%-%ds %%s' % (2+len(maxlen_str))
print("@menu")
for cmd in sorted(list(commands)):
print(fmt % ("%s::" % cmd, "Undocumented command"))
print("@end menu")
print()
for cmd in sorted(list(commands)):
print("@node %s" % cmd)
print("@subsection %s" % cmd)
print()
print("The grub command @command{%s} has not been documented yet." % cmd)
print()
def print_set(st):
for i, item in enumerate(sorted(list(st)), start=1):
print("@comment", " %d." % i, item)
def main():
cc = CommandChecker()
print("@comment",
"%s - compare documented and implemented commands" % cc.prog)
# texi_text_commands = cc.read_texi_text_commands('grub.texi')
# print("@comment", "Commands in grub.texi text:")
# print_set(texi_text_commands)
texi_menu_commands = cc.read_texi_command_menu('grub.texi')
# print("@comment", "Commands in grub.texi menu:")
# print_set(texi_menu_commands)
src_commands = cc.read_src_commands()
# print("@comment", "Commands in grub source:")
# print_set(src_commands)
doc_without_src = texi_menu_commands - src_commands
print("@comment",
"Commands documented in a grub.texi menu but not registered in grub source:")
print_set(doc_without_src)
src_without_doc = src_commands - texi_menu_commands
print("@comment",
"Commands registered in grub source but not documented in a grub.texi node:")
print_set(src_without_doc)
print()
write_undocumented_commands(src_without_doc)
if ((len(doc_without_src) > 0) or (len(src_without_doc) > 0)):
sys.exit(1)
sys.exit(0)
if __name__ == '__main__':
main()
####################################################################
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: 119 grub commands not documented in grub.texi
2020-04-18 10:53 119 grub commands not documented in grub.texi Hans Ulrich Niedermann
@ 2020-04-22 10:10 ` Daniel Kiper
2020-04-22 14:35 ` Eli Schwartz
2020-04-22 19:29 ` Hans Ulrich Niedermann
0 siblings, 2 replies; 9+ messages in thread
From: Daniel Kiper @ 2020-04-22 10:10 UTC (permalink / raw)
To: Hans Ulrich Niedermann; +Cc: grub-devel
Hi Hans,
On Sat, Apr 18, 2020 at 12:53:12PM +0200, Hans Ulrich Niedermann wrote:
> Hi,
>
> I have noticed that there are two commands documented in grub.texi
> which appear not to occur anywhere within the grub source code:
> 'pxe_unload' and 'uppermem'.
>
> @node pxe_unload
> @subsection pxe_unload
>
> @deffn Command pxe_unload
> Unload the PXE environment (@pxref{Network}).
>
> This command is only available on PC BIOS systems.
> @end deffn
>
> However, judging from the source code ("git grep pxe_unload"),
> pxe_unload is not available at all, so the documentation is wrong when
> it claims that the command were available on PC BIOS systems.
>
> Should 'pxe_unload' be removed from grub.texi, or does it need a notice
> that there are plans to implement it?
If it is not implemented please drop it.
> Regarding 'uppermem', there is only
>
> @node uppermem
> @subsection uppermem
>
> This command is not yet implemented for GRUB 2, although it is planned.
>
> This may be a valid state for the "uppermem" command.
Probably it is a remnant from ancient times. Please drop it too.
> So far for those two commands.
>
> Then, a bit more surprisingly, there appear to be 119 (yes, more than
> 100) grub commands which are registered within the grub source code but
> are not documented within grub.texi.
>
> * When do I count a command as documented in grub.texi?
> Every @node/@subsection in a "@section * commands" of
> "@chapter Commands".
>
> * What do I count as a command registered within grub source?
> Whatever string appears as the first argument in a call to any of
> the following functions:
>
> * grub_register_command
> * grub_register_command_p1
> * grub_register_extcmd
At first sight it makes sense...
> Some of these undocumented commands look like they could be relatively
> easily be grouped into a @section and described there (e.g. the 'break',
> 'continue', 'exit', 'return' commands for a group of shell like
> commands, maybe together with '[').
OK...
> I am absolutely certain that I will not find the time to write useful
> documentation for a significant portion of these undocumented commands
> within the grub 2.06 release timeframe (i.e. until June 2020), or even
> just determine which ones I think actually need to or should be
> documented publically.
Sure thing! This can be long term effort if you want to do that.
> So if someone wants their favourite commands documented for 2.06, they
> will need to write that documentation themselves. I will just try to
> add tentative documentation for @command{module2} and
> @command{multiboot2} so that booting of a multiboot2 kernel will at
> least be mentioned in grub.texi at all.
Works for me...
> If you want me to, I can submit a patch for the check-commands.py
I cannot find check-commands.py, anyway...
> script to autogenerate a list of undocumented commands which should be
> documented which can then be auto-included in grub.texi and appear in
> the generated 'grub.info'. This would give a reader of the grub manual
> at least the knowledge that those command actually exist, and could be
> coupled with a call for help documenting those commands right inside
> grub.texi itself.
That would be perfect. Especially if it appears in 2.06 release.
Thank you for doing this work!
Daniel
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: 119 grub commands not documented in grub.texi
2020-04-22 10:10 ` Daniel Kiper
@ 2020-04-22 14:35 ` Eli Schwartz
2020-04-22 19:29 ` Hans Ulrich Niedermann
1 sibling, 0 replies; 9+ messages in thread
From: Eli Schwartz @ 2020-04-22 14:35 UTC (permalink / raw)
To: grub-devel
[-- Attachment #1.1: Type: text/plain, Size: 934 bytes --]
On 4/22/20 6:10 AM, Daniel Kiper wrote:
>> If you want me to, I can submit a patch for the check-commands.py
>
> I cannot find check-commands.py, anyway...
>
>> script to autogenerate a list of undocumented commands which should be
>> documented which can then be auto-included in grub.texi and appear in
>> the generated 'grub.info'. This would give a reader of the grub manual
>> at least the knowledge that those command actually exist, and could be
>> coupled with a call for help documenting those commands right inside
>> grub.texi itself.
>
> That would be perfect. Especially if it appears in 2.06 release.
>
> Thank you for doing this work!
I believe the suggestion is to add the new program check-commands.py
provided at the end of the previous message, AND make it autogenerate a
grub.texi stub listing "not-yet-documented commands".
--
Eli Schwartz
Arch Linux Bug Wrangler and Trusted User
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 1601 bytes --]
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: 119 grub commands not documented in grub.texi
2020-04-22 10:10 ` Daniel Kiper
2020-04-22 14:35 ` Eli Schwartz
@ 2020-04-22 19:29 ` Hans Ulrich Niedermann
2020-04-22 19:32 ` [PATCH 1/3] docs: Add grub command documentation checks Hans Ulrich Niedermann
2020-04-23 10:30 ` 119 grub commands not documented in grub.texi Daniel Kiper
1 sibling, 2 replies; 9+ messages in thread
From: Hans Ulrich Niedermann @ 2020-04-22 19:29 UTC (permalink / raw)
To: grub-devel
On Wed, 22 Apr 2020 12:10:31 +0200
Daniel Kiper <dkiper@net-space.pl> wrote:
> On Sat, Apr 18, 2020 at 12:53:12PM +0200, Hans Ulrich Niedermann
> wrote:
> > I have noticed that there are two commands documented in grub.texi
> > which appear not to occur anywhere within the grub source code:
> > 'pxe_unload' and 'uppermem'.
> >
> > @node pxe_unload
> > @subsection pxe_unload
> >
> > @deffn Command pxe_unload
> > Unload the PXE environment (@pxref{Network}).
> >
> > This command is only available on PC BIOS systems.
> > @end deffn
> >
> > However, judging from the source code ("git grep pxe_unload"),
> > pxe_unload is not available at all, so the documentation is wrong
> > when it claims that the command were available on PC BIOS systems.
> >
> > Should 'pxe_unload' be removed from grub.texi, or does it need a
> > notice that there are plans to implement it?
>
> If it is not implemented please drop it.
I could not find any traces in the current source tree: The string
pxe_unload only appears in ChangeLog-2015 and docs/grub.texi:
$ git grep pxe_unload
ChangeLog-2015: * include/grub/i386/pc/pxe.h (grub_pxe_unload): Removed.
ChangeLog-2015: (pxe_unload): New section.
ChangeLog-2015: (grub_cmd_pxe_unload): New function.
docs/grub.texi:* pxe_unload:: Unload the PXE environment
docs/grub.texi:@node pxe_unload
docs/grub.texi:@subsection pxe_unload
docs/grub.texi:@deffn Command pxe_unload
$
Looks like 671a78acb0648d3d73c95ab0f021f907499aacc0 from 2011-07-05
removed the code. The removal of the 'pxe_unload' documentation will be
in my patch set.
> > Regarding 'uppermem', there is only
> >
> > @node uppermem
> > @subsection uppermem
> >
> > This command is not yet implemented for GRUB 2, although it is
> > planned.
> >
> > This may be a valid state for the "uppermem" command.
>
> Probably it is a remnant from ancient times. Please drop it too.
I have found the (non-existing) 'uppermem' command mentioned in the
'GNU/Linux' node (info grub -n 'GNU/Linux') as well:
| @strong{Caution:} If you use an initrd and specify the @samp{mem=}
| option to the kernel to let it use less than actual memory size, you
| will also have to specify the same memory size to GRUB. To let GRUB
| know the size, run the command @command{uppermem} @emph{before}
| loading the kernel. @xref{uppermem}, for more information.
Maybe someone with some familiarity with the code which implements the
"linux" and "initrd" commands and how Linux kernels and Linux
initrds are loaded could shed some light on the current status of
the 'mem=' Linux kernel parameter and the non-existing 'uppermem' grub
command.
Might this be the reason why 'uppermem' has been planned for Grub2 in
the past?
> > If you want me to, I can submit a patch for the check-commands.py
>
> I cannot find check-commands.py, anyway...
>
> > script to autogenerate a list of undocumented commands which should
> > be documented which can then be auto-included in grub.texi and
> > appear in the generated 'grub.info'. This would give a reader of
> > the grub manual at least the knowledge that those command actually
> > exist, and could be coupled with a call for help documenting those
> > commands right inside grub.texi itself.
>
> That would be perfect. Especially if it appears in 2.06 release.
That patch is on its way.
Uli
^ permalink raw reply [flat|nested] 9+ messages in thread
* [PATCH 1/3] docs: Add grub command documentation checks
2020-04-22 19:29 ` Hans Ulrich Niedermann
@ 2020-04-22 19:32 ` Hans Ulrich Niedermann
2020-04-22 19:32 ` [PATCH 2/3] docs: Remove docs for non-existing pxe_unload command Hans Ulrich Niedermann
2020-04-22 19:42 ` [PATCH 1/3 v2] docs: Add grub command documentation checks Hans Ulrich Niedermann
2020-04-23 10:30 ` 119 grub commands not documented in grub.texi Daniel Kiper
1 sibling, 2 replies; 9+ messages in thread
From: Hans Ulrich Niedermann @ 2020-04-22 19:32 UTC (permalink / raw)
To: grub-devel; +Cc: Hans Ulrich Niedermann
This adds the docs/check-commands.py script which checks whether
grub commands implemented in the git source tree are documented
in docs/grub.texi and vice versa.
During a standard "make" command, BUILT_SOURCES makes sure that
docs/undocumented-commands.texi will be created (if it does not
exist yet) or updated (if it differs from what the script would
generate) or be left alone otherwise.
If you run "make grub.info" in the docs/ subdirectory, the
BUILT_SOURCES trick to create or update undocumented-commands.texi
will not work (this is an Automake limitation). So if you want
to avoid the "all" make target, you need to explictly run
"make update-undoc" first.
The build time requirements are already required by other parts
of the grub buildsystem:
* $(CMP)
compare two files by content
* $(PYTHON)
check-commands.py has been written to run on Python 2 and 3
(tested with Python 2.7 and Python 3.7)
The time docs/check-commands.py needs to read and basically grep
through docs/grub.texi and all *.c source files is quick enough
to not significantly slow down a "make" on the grub source tree.
Signed-off-by: Hans Ulrich Niedermann <hun@n-dimensional.de>
---
.gitignore | 1 +
Makefile.am | 2 +-
docs/Makefile.am | 16 +++-
docs/check-commands.py | 173 +++++++++++++++++++++++++++++++++++++++++
docs/grub.texi | 4 +
5 files changed, 192 insertions(+), 4 deletions(-)
create mode 100644 docs/check-commands.py
diff --git a/.gitignore b/.gitignore
index 149b3713a..bdedd4e89 100644
--- a/.gitignore
+++ b/.gitignore
@@ -35,6 +35,7 @@ DISTLIST
docs/*.info
/docs/*.info-*
docs/stamp-vti
+/docs/undocumented-commands.texi
docs/version.texi
ehci_test
example_grub_script_test
diff --git a/Makefile.am b/Makefile.am
index bf9c1ba64..264e3f659 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -412,7 +412,7 @@ endif
.PHONY: bootcheck-linux-i386 bootcheck-linux-x86_64 \
bootcheck-kfreebsd-i386 bootcheck-kfreebsd-x86_64 \
bootcheck-knetbsd-i386 bootcheck-knetbsd-x86_64 \
- bootcheck-linux-mips FORCE
+ bootcheck-linux-mips FORCE update-undoc
# Randomly generated
SUCCESSFUL_BOOT_STRING=3e49994fd5d82b7c9298d672d774080d
diff --git a/docs/Makefile.am b/docs/Makefile.am
index 93eb39627..8e8500226 100644
--- a/docs/Makefile.am
+++ b/docs/Makefile.am
@@ -2,8 +2,18 @@ AUTOMAKE_OPTIONS = subdir-objects
# AM_MAKEINFOFLAGS = --no-split --no-validate
info_TEXINFOS = grub.texi grub-dev.texi
-grub_TEXINFOS = fdl.texi
-
-EXTRA_DIST = font_char_metrics.png font_char_metrics.txt
+grub_TEXINFOS = fdl.texi undocumented-commands.texi
+EXTRA_DIST = check-commands.py font_char_metrics.png font_char_metrics.txt
+CLEANFILES = undocumented-commands.texi
+BUILT_SOURCES = update-undoc
+update-undoc:
+ $(PYTHON) check-commands.py > undocumented-commands.texi.tmp
+ @if test -f $(srcdir)/undocumented-commands.texi && $(CMP) undocumented-commands.texi.tmp $(srcdir)/undocumented-commands.texi; then \
+ echo "Not updating undocumented-commands.texi: is up to date"; \
+ rm -f undocumented-commands.texi.tmp; \
+ else \
+ echo "Updating undocumented-commands.texi"; \
+ mv -f undocumented-commands.texi.tmp $(srcdir)/undocumented-commands.texi; \
+ fi
diff --git a/docs/check-commands.py b/docs/check-commands.py
new file mode 100644
index 000000000..3c99bfbb2
--- /dev/null
+++ b/docs/check-commands.py
@@ -0,0 +1,173 @@
+#!/usr/bin/env python
+
+from __future__ import print_function
+
+import os
+import re
+
+import sys
+
+
+class CommandChecker:
+
+ def __init__(self):
+ srcdir, self.prog = os.path.split(__file__)
+ self.top_srcdir = os.path.dirname(os.path.abspath(srcdir))
+
+ def read_texi_text_commands(self, texi_filename):
+ texi_pathname = os.path.join(self.top_srcdir, 'docs', texi_filename)
+ r = re.compile(r'@command\{([a-zA-Z0-9_-]+)\}')
+ commands = set()
+ with open(texi_pathname) as texi:
+ for line in texi.readlines():
+ for m in r.finditer(line):
+ commands.add(m[1])
+ return commands
+
+ def read_texi_command_menu(self, texi_filename):
+ texi_pathname = os.path.join(self.top_srcdir, 'docs', texi_filename)
+ r = re.compile(r'\* ([^:]+)::')
+ n = re.compile(r'^@node .+ commands$')
+ commands = set()
+ waiting_for_node = True
+ waiting_for_menu = False
+ collecting_commands = False
+ with open(texi_pathname) as texi:
+ for line in texi.readlines():
+ line = line.strip()
+ if line.startswith('@comment'):
+ continue
+ if waiting_for_node:
+ if n.match(line):
+ # print("@comment", line)
+ waiting_for_node = False
+ waiting_for_menu = True
+ continue
+ if waiting_for_menu:
+ if line == '@menu':
+ waiting_for_menu = False
+ collecting_commands = True
+ continue
+ if collecting_commands:
+ if line == '@end menu':
+ collecting_commands = False
+ waiting_for_node = True
+ continue
+ m = r.match(line)
+ # print("@comment ", m[1])
+ commands.add(m.group(1))
+ return commands
+
+ def read_src_commands(self):
+ top = os.path.join(self.top_srcdir, 'grub-core')
+ r = re.compile(r'grub_register_(command|command_p1|extcmd)\s*\("([a-z0-9A-Z_\[]+)",')
+ commands = set()
+ for dirpath, dirnames, filenames in os.walk(top):
+ for fn in filenames:
+ fp = os.path.join(dirpath, fn)
+ fpe = os.path.splitext(fp)[1]
+ if fpe == '.c':
+ with open(fp) as cfile:
+ for line in cfile.readlines():
+ for m in r.finditer(line):
+ commands.add(m.group(2))
+ return commands
+
+
+def write_undocumented_commands(commands):
+ print("""\
+@node Undocumented commands
+@section The list of undocumented commands
+""")
+
+ if not commands:
+ print("""
+There appear to be no undocumented commands at this time.
+""")
+ return
+
+ print("""\
+These commands are implemented in the grub software but still need to be
+documented and sorted into categories.
+""")
+
+ maxlen_str = sorted(list(commands), key=lambda cmd: len(cmd),
+ reverse=True)[0]
+ fmt = '* %%-%ds %%s' % (2+len(maxlen_str))
+
+ print("@menu")
+ for cmd in sorted(list(commands)):
+ print(fmt % ("%s::" % cmd, "Undocumented command"))
+ print("@end menu")
+ print()
+
+ for cmd in sorted(list(commands)):
+ print("@node %s" % cmd)
+ print("@subsection %s" % cmd)
+ print()
+ print("The grub command @command{%s} has not been documented properly yet." % cmd)
+ print()
+
+
+def print_set(st):
+ for i, item in enumerate(sorted(list(st)), start=1):
+ print("@comment", " %d." % i, item)
+
+
+def main():
+ cc = CommandChecker()
+ print("""\
+@comment Automatically generated by %(prog)s
+@comment Do not modify this generated file.
+@comment
+@comment If you want to document some of the commands listed below, feel
+@comment free to copy over some of the @nodes and @subsections below
+@comment into the grub.texi file proper and re-generate this file.
+@comment""" % {'prog': cc.prog})
+
+ # texi_text_commands = cc.read_texi_text_commands('grub.texi')
+ # print("@comment", "Commands in grub.texi text:")
+ # print_set(texi_text_commands)
+
+ texi_menu_commands = cc.read_texi_command_menu('grub.texi')
+ # print("@comment", "Commands in grub.texi menu:")
+ # print_set(texi_menu_commands)
+
+ src_commands = cc.read_src_commands()
+ # print("@comment", "Commands in grub source:")
+ # print_set(src_commands)
+
+ doc_without_src = texi_menu_commands - src_commands
+ if doc_without_src:
+ print("@comment",
+ "Commands documented in a grub.texi menu but not registered in grub source:")
+ print_set(doc_without_src)
+ else:
+ print("@comment",
+ "There appear to be no commands documented in a grub.texi menu but not")
+ print("@comment",
+ "registered in grub source. Congratulations!")
+
+ print("@comment")
+
+ src_without_doc = src_commands - texi_menu_commands
+ if src_without_doc:
+ print("@comment",
+ "Commands registered in grub source but not documented in a grub.texi node:")
+ print_set(src_without_doc)
+
+ print()
+
+ write_undocumented_commands(src_without_doc)
+
+ # Once grub.texi actually documents all commands, we can uncomment
+ # this and actually fail if the set of implemented commands and
+ # the set of documented commands differ in any way.
+ # if ((len(doc_without_src) > 0) or (len(src_without_doc) > 0)):
+ # sys.exit(1)
+
+ sys.exit(0)
+
+
+if __name__ == '__main__':
+ main()
diff --git a/docs/grub.texi b/docs/grub.texi
index d6408d242..0865ffa17 100644
--- a/docs/grub.texi
+++ b/docs/grub.texi
@@ -3764,6 +3764,7 @@ shell}.
* General commands::
* Command-line and menu entry commands::
* Networking commands::
+* Undocumented commands::
@end menu
@@ -5636,6 +5637,9 @@ is given, use default list of servers.
@end deffn
+@include undocumented-commands.texi
+
+
@node Internationalisation
@chapter Internationalisation
--
2.25.3
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH 2/3] docs: Remove docs for non-existing pxe_unload command
2020-04-22 19:32 ` [PATCH 1/3] docs: Add grub command documentation checks Hans Ulrich Niedermann
@ 2020-04-22 19:32 ` Hans Ulrich Niedermann
2020-04-23 10:16 ` Daniel Kiper
2020-04-22 19:42 ` [PATCH 1/3 v2] docs: Add grub command documentation checks Hans Ulrich Niedermann
1 sibling, 1 reply; 9+ messages in thread
From: Hans Ulrich Niedermann @ 2020-04-22 19:32 UTC (permalink / raw)
To: grub-devel; +Cc: Hans Ulrich Niedermann
Remove the documentation for the pxe_unload command from
the docs/grub.texi file.
The pxe_unload command is not implemented in the grub
source at all and appears to have been removed in commit
71a78acb0648d3d73c95ab0f021f907499aacc0 (from 2011-07-05).
Signed-off-by: Hans Ulrich Niedermann <hun@n-dimensional.de>
---
docs/grub.texi | 11 -----------
1 file changed, 11 deletions(-)
diff --git a/docs/grub.texi b/docs/grub.texi
index 0865ffa17..0f7886bb6 100644
--- a/docs/grub.texi
+++ b/docs/grub.texi
@@ -4012,7 +4012,6 @@ you forget a command, you can run the command @command{help}
* password_pbkdf2:: Set a hashed password
* play:: Play a tune
* probe:: Retrieve device info
-* pxe_unload:: Unload the PXE environment
* rdmsr:: Read values from model-specific registers
* read:: Read user input
* reboot:: Reboot your computer
@@ -4863,16 +4862,6 @@ The option @option{--part-uuid} is currently only implemented for GPT-formatted
@end deffn
-@node pxe_unload
-@subsection pxe_unload
-
-@deffn Command pxe_unload
-Unload the PXE environment (@pxref{Network}).
-
-This command is only available on PC BIOS systems.
-@end deffn
-
-
@node rdmsr
@subsection rdmsr
--
2.25.3
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH 1/3 v2] docs: Add grub command documentation checks
2020-04-22 19:32 ` [PATCH 1/3] docs: Add grub command documentation checks Hans Ulrich Niedermann
2020-04-22 19:32 ` [PATCH 2/3] docs: Remove docs for non-existing pxe_unload command Hans Ulrich Niedermann
@ 2020-04-22 19:42 ` Hans Ulrich Niedermann
1 sibling, 0 replies; 9+ messages in thread
From: Hans Ulrich Niedermann @ 2020-04-22 19:42 UTC (permalink / raw)
To: grub-devel; +Cc: Hans Ulrich Niedermann
This adds the docs/check-commands.py script which checks whether
grub commands implemented in the git source tree are documented
in docs/grub.texi and vice versa.
During a standard "make" command, BUILT_SOURCES makes sure that
docs/undocumented-commands.texi will be created (if it does not
exist yet) or updated (if it differs from what the script would
generate) or be left alone otherwise.
If you run "make grub.info" in the docs/ subdirectory, the
BUILT_SOURCES trick to create or update undocumented-commands.texi
will not work (this is an Automake limitation). So if you want
to avoid the "all" make target, you need to explictly run
"make update-undoc" first.
The build time requirements are already required by other parts
of the grub buildsystem:
* $(CMP)
compare two files by content
* $(PYTHON)
check-commands.py has been written to run on Python 2 and 3
(tested with Python 2.7 and Python 3.7)
The time docs/check-commands.py needs to read and basically grep
through docs/grub.texi and all *.c source files is quick enough
to not significantly slow down a "make" on the grub source tree.
Signed-off-by: Hans Ulrich Niedermann <hun@n-dimensional.de>
---
.gitignore | 1 +
docs/Makefile.am | 16 +++-
docs/check-commands.py | 173 +++++++++++++++++++++++++++++++++++++++++
docs/grub.texi | 4 +
4 files changed, 191 insertions(+), 3 deletions(-)
create mode 100644 docs/check-commands.py
diff --git a/.gitignore b/.gitignore
index 149b3713a..bdedd4e89 100644
--- a/.gitignore
+++ b/.gitignore
@@ -35,6 +35,7 @@ DISTLIST
docs/*.info
/docs/*.info-*
docs/stamp-vti
+/docs/undocumented-commands.texi
docs/version.texi
ehci_test
example_grub_script_test
diff --git a/docs/Makefile.am b/docs/Makefile.am
index 93eb39627..8e8500226 100644
--- a/docs/Makefile.am
+++ b/docs/Makefile.am
@@ -2,8 +2,18 @@ AUTOMAKE_OPTIONS = subdir-objects
# AM_MAKEINFOFLAGS = --no-split --no-validate
info_TEXINFOS = grub.texi grub-dev.texi
-grub_TEXINFOS = fdl.texi
-
-EXTRA_DIST = font_char_metrics.png font_char_metrics.txt
+grub_TEXINFOS = fdl.texi undocumented-commands.texi
+EXTRA_DIST = check-commands.py font_char_metrics.png font_char_metrics.txt
+CLEANFILES = undocumented-commands.texi
+BUILT_SOURCES = update-undoc
+update-undoc:
+ $(PYTHON) check-commands.py > undocumented-commands.texi.tmp
+ @if test -f $(srcdir)/undocumented-commands.texi && $(CMP) undocumented-commands.texi.tmp $(srcdir)/undocumented-commands.texi; then \
+ echo "Not updating undocumented-commands.texi: is up to date"; \
+ rm -f undocumented-commands.texi.tmp; \
+ else \
+ echo "Updating undocumented-commands.texi"; \
+ mv -f undocumented-commands.texi.tmp $(srcdir)/undocumented-commands.texi; \
+ fi
diff --git a/docs/check-commands.py b/docs/check-commands.py
new file mode 100644
index 000000000..3c99bfbb2
--- /dev/null
+++ b/docs/check-commands.py
@@ -0,0 +1,173 @@
+#!/usr/bin/env python
+
+from __future__ import print_function
+
+import os
+import re
+
+import sys
+
+
+class CommandChecker:
+
+ def __init__(self):
+ srcdir, self.prog = os.path.split(__file__)
+ self.top_srcdir = os.path.dirname(os.path.abspath(srcdir))
+
+ def read_texi_text_commands(self, texi_filename):
+ texi_pathname = os.path.join(self.top_srcdir, 'docs', texi_filename)
+ r = re.compile(r'@command\{([a-zA-Z0-9_-]+)\}')
+ commands = set()
+ with open(texi_pathname) as texi:
+ for line in texi.readlines():
+ for m in r.finditer(line):
+ commands.add(m[1])
+ return commands
+
+ def read_texi_command_menu(self, texi_filename):
+ texi_pathname = os.path.join(self.top_srcdir, 'docs', texi_filename)
+ r = re.compile(r'\* ([^:]+)::')
+ n = re.compile(r'^@node .+ commands$')
+ commands = set()
+ waiting_for_node = True
+ waiting_for_menu = False
+ collecting_commands = False
+ with open(texi_pathname) as texi:
+ for line in texi.readlines():
+ line = line.strip()
+ if line.startswith('@comment'):
+ continue
+ if waiting_for_node:
+ if n.match(line):
+ # print("@comment", line)
+ waiting_for_node = False
+ waiting_for_menu = True
+ continue
+ if waiting_for_menu:
+ if line == '@menu':
+ waiting_for_menu = False
+ collecting_commands = True
+ continue
+ if collecting_commands:
+ if line == '@end menu':
+ collecting_commands = False
+ waiting_for_node = True
+ continue
+ m = r.match(line)
+ # print("@comment ", m[1])
+ commands.add(m.group(1))
+ return commands
+
+ def read_src_commands(self):
+ top = os.path.join(self.top_srcdir, 'grub-core')
+ r = re.compile(r'grub_register_(command|command_p1|extcmd)\s*\("([a-z0-9A-Z_\[]+)",')
+ commands = set()
+ for dirpath, dirnames, filenames in os.walk(top):
+ for fn in filenames:
+ fp = os.path.join(dirpath, fn)
+ fpe = os.path.splitext(fp)[1]
+ if fpe == '.c':
+ with open(fp) as cfile:
+ for line in cfile.readlines():
+ for m in r.finditer(line):
+ commands.add(m.group(2))
+ return commands
+
+
+def write_undocumented_commands(commands):
+ print("""\
+@node Undocumented commands
+@section The list of undocumented commands
+""")
+
+ if not commands:
+ print("""
+There appear to be no undocumented commands at this time.
+""")
+ return
+
+ print("""\
+These commands are implemented in the grub software but still need to be
+documented and sorted into categories.
+""")
+
+ maxlen_str = sorted(list(commands), key=lambda cmd: len(cmd),
+ reverse=True)[0]
+ fmt = '* %%-%ds %%s' % (2+len(maxlen_str))
+
+ print("@menu")
+ for cmd in sorted(list(commands)):
+ print(fmt % ("%s::" % cmd, "Undocumented command"))
+ print("@end menu")
+ print()
+
+ for cmd in sorted(list(commands)):
+ print("@node %s" % cmd)
+ print("@subsection %s" % cmd)
+ print()
+ print("The grub command @command{%s} has not been documented properly yet." % cmd)
+ print()
+
+
+def print_set(st):
+ for i, item in enumerate(sorted(list(st)), start=1):
+ print("@comment", " %d." % i, item)
+
+
+def main():
+ cc = CommandChecker()
+ print("""\
+@comment Automatically generated by %(prog)s
+@comment Do not modify this generated file.
+@comment
+@comment If you want to document some of the commands listed below, feel
+@comment free to copy over some of the @nodes and @subsections below
+@comment into the grub.texi file proper and re-generate this file.
+@comment""" % {'prog': cc.prog})
+
+ # texi_text_commands = cc.read_texi_text_commands('grub.texi')
+ # print("@comment", "Commands in grub.texi text:")
+ # print_set(texi_text_commands)
+
+ texi_menu_commands = cc.read_texi_command_menu('grub.texi')
+ # print("@comment", "Commands in grub.texi menu:")
+ # print_set(texi_menu_commands)
+
+ src_commands = cc.read_src_commands()
+ # print("@comment", "Commands in grub source:")
+ # print_set(src_commands)
+
+ doc_without_src = texi_menu_commands - src_commands
+ if doc_without_src:
+ print("@comment",
+ "Commands documented in a grub.texi menu but not registered in grub source:")
+ print_set(doc_without_src)
+ else:
+ print("@comment",
+ "There appear to be no commands documented in a grub.texi menu but not")
+ print("@comment",
+ "registered in grub source. Congratulations!")
+
+ print("@comment")
+
+ src_without_doc = src_commands - texi_menu_commands
+ if src_without_doc:
+ print("@comment",
+ "Commands registered in grub source but not documented in a grub.texi node:")
+ print_set(src_without_doc)
+
+ print()
+
+ write_undocumented_commands(src_without_doc)
+
+ # Once grub.texi actually documents all commands, we can uncomment
+ # this and actually fail if the set of implemented commands and
+ # the set of documented commands differ in any way.
+ # if ((len(doc_without_src) > 0) or (len(src_without_doc) > 0)):
+ # sys.exit(1)
+
+ sys.exit(0)
+
+
+if __name__ == '__main__':
+ main()
diff --git a/docs/grub.texi b/docs/grub.texi
index d6408d242..0865ffa17 100644
--- a/docs/grub.texi
+++ b/docs/grub.texi
@@ -3764,6 +3764,7 @@ shell}.
* General commands::
* Command-line and menu entry commands::
* Networking commands::
+* Undocumented commands::
@end menu
@@ -5636,6 +5637,9 @@ is given, use default list of servers.
@end deffn
+@include undocumented-commands.texi
+
+
@node Internationalisation
@chapter Internationalisation
--
2.25.3
^ permalink raw reply related [flat|nested] 9+ messages in thread
* Re: [PATCH 2/3] docs: Remove docs for non-existing pxe_unload command
2020-04-22 19:32 ` [PATCH 2/3] docs: Remove docs for non-existing pxe_unload command Hans Ulrich Niedermann
@ 2020-04-23 10:16 ` Daniel Kiper
0 siblings, 0 replies; 9+ messages in thread
From: Daniel Kiper @ 2020-04-23 10:16 UTC (permalink / raw)
To: Hans Ulrich Niedermann; +Cc: grub-devel
On Wed, Apr 22, 2020 at 09:32:41PM +0200, Hans Ulrich Niedermann wrote:
> Remove the documentation for the pxe_unload command from
> the docs/grub.texi file.
>
> The pxe_unload command is not implemented in the grub
> source at all and appears to have been removed in commit
> 71a78acb0648d3d73c95ab0f021f907499aacc0 (from 2011-07-05).
>
> Signed-off-by: Hans Ulrich Niedermann <hun@n-dimensional.de>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
Daniel
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: 119 grub commands not documented in grub.texi
2020-04-22 19:29 ` Hans Ulrich Niedermann
2020-04-22 19:32 ` [PATCH 1/3] docs: Add grub command documentation checks Hans Ulrich Niedermann
@ 2020-04-23 10:30 ` Daniel Kiper
1 sibling, 0 replies; 9+ messages in thread
From: Daniel Kiper @ 2020-04-23 10:30 UTC (permalink / raw)
To: Hans Ulrich Niedermann; +Cc: grub-devel
On Wed, Apr 22, 2020 at 09:29:13PM +0200, Hans Ulrich Niedermann wrote:
> On Wed, 22 Apr 2020 12:10:31 +0200
> Daniel Kiper <dkiper@net-space.pl> wrote:
>
> > On Sat, Apr 18, 2020 at 12:53:12PM +0200, Hans Ulrich Niedermann
> > wrote:
> > > I have noticed that there are two commands documented in grub.texi
> > > which appear not to occur anywhere within the grub source code:
> > > 'pxe_unload' and 'uppermem'.
> > >
> > > @node pxe_unload
> > > @subsection pxe_unload
> > >
> > > @deffn Command pxe_unload
> > > Unload the PXE environment (@pxref{Network}).
> > >
> > > This command is only available on PC BIOS systems.
> > > @end deffn
> > >
> > > However, judging from the source code ("git grep pxe_unload"),
> > > pxe_unload is not available at all, so the documentation is wrong
> > > when it claims that the command were available on PC BIOS systems.
> > >
> > > Should 'pxe_unload' be removed from grub.texi, or does it need a
> > > notice that there are plans to implement it?
> >
> > If it is not implemented please drop it.
>
> I could not find any traces in the current source tree: The string
> pxe_unload only appears in ChangeLog-2015 and docs/grub.texi:
>
> $ git grep pxe_unload
> ChangeLog-2015: * include/grub/i386/pc/pxe.h (grub_pxe_unload): Removed.
> ChangeLog-2015: (pxe_unload): New section.
> ChangeLog-2015: (grub_cmd_pxe_unload): New function.
> docs/grub.texi:* pxe_unload:: Unload the PXE environment
> docs/grub.texi:@node pxe_unload
> docs/grub.texi:@subsection pxe_unload
> docs/grub.texi:@deffn Command pxe_unload
> $
>
> Looks like 671a78acb0648d3d73c95ab0f021f907499aacc0 from 2011-07-05
> removed the code. The removal of the 'pxe_unload' documentation will be
> in my patch set.
>
> > > Regarding 'uppermem', there is only
> > >
> > > @node uppermem
> > > @subsection uppermem
> > >
> > > This command is not yet implemented for GRUB 2, although it is
> > > planned.
> > >
> > > This may be a valid state for the "uppermem" command.
> >
> > Probably it is a remnant from ancient times. Please drop it too.
>
> I have found the (non-existing) 'uppermem' command mentioned in the
> 'GNU/Linux' node (info grub -n 'GNU/Linux') as well:
>
> | @strong{Caution:} If you use an initrd and specify the @samp{mem=}
> | option to the kernel to let it use less than actual memory size, you
> | will also have to specify the same memory size to GRUB. To let GRUB
> | know the size, run the command @command{uppermem} @emph{before}
> | loading the kernel. @xref{uppermem}, for more information.
>
> Maybe someone with some familiarity with the code which implements the
> "linux" and "initrd" commands and how Linux kernels and Linux
> initrds are loaded could shed some light on the current status of
> the 'mem=' Linux kernel parameter and the non-existing 'uppermem' grub
> command.
>
> Might this be the reason why 'uppermem' has been planned for Grub2 in
> the past?
It looks that it was planned. However, if nobody wrote it up until now
then nobody wants to use it. So, I think that we can safely drop it from
the documentation to not confuse the user.
> > > If you want me to, I can submit a patch for the check-commands.py
> >
> > I cannot find check-commands.py, anyway...
> >
> > > script to autogenerate a list of undocumented commands which should
> > > be documented which can then be auto-included in grub.texi and
> > > appear in the generated 'grub.info'. This would give a reader of
> > > the grub manual at least the knowledge that those command actually
> > > exist, and could be coupled with a call for help documenting those
> > > commands right inside grub.texi itself.
> >
> > That would be perfect. Especially if it appears in 2.06 release.
>
> That patch is on its way.
I saw it. However, I have to ask somebody more familiar with python than
I to make its review.
Daniel
^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2020-04-23 10:47 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-04-18 10:53 119 grub commands not documented in grub.texi Hans Ulrich Niedermann
2020-04-22 10:10 ` Daniel Kiper
2020-04-22 14:35 ` Eli Schwartz
2020-04-22 19:29 ` Hans Ulrich Niedermann
2020-04-22 19:32 ` [PATCH 1/3] docs: Add grub command documentation checks Hans Ulrich Niedermann
2020-04-22 19:32 ` [PATCH 2/3] docs: Remove docs for non-existing pxe_unload command Hans Ulrich Niedermann
2020-04-23 10:16 ` Daniel Kiper
2020-04-22 19:42 ` [PATCH 1/3 v2] docs: Add grub command documentation checks Hans Ulrich Niedermann
2020-04-23 10:30 ` 119 grub commands not documented in grub.texi Daniel Kiper
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.