[PATCH v5 00/14] GDB script fixes and improvements

[PATCH v5 00/14] GDB script fixes and improvements

[Glenn Washburn]

This series has been substantially rewritten since v4, although a large
minority of the patches haven't changed much. The biggest change is
that the implementation has been converted to Python instead of what was
done in shell script. I have always felt it should be done in Python, but
it seemed daunting to learn the Python-GDB API, so the shell script seemed
the path of least resistance. I decided to give it a second look and was
surprised it wasn't as bad as I thought it would be. Because the python API
is so tightly integrated into GDB, there are things you can do with it that
either aren't possible or which look like ugly hacks when attempting with
the native script.

The other big change is that there is a new --enable-efi-debug option to
configure which enables the printing of the gdb command for loading the
GRUB kernel symbols. This is disallowed when booting with Secure Boot on.
There are two ways the GDB command is printed: (1) in early EFI setup and
(2) on-demand by a user using the gdbinfo command.

There are a couple new features to the GDB script, like a trivial one that
changes the gdb prompt and another that allows for software breakpoints to
be set before the GRUB image is loaded.

This series also incorporates suggestions given for v4 and adds more to the
documentation.

Glenn

Glenn Washburn (14):
  gdb: Fix redirection issue in dump_module_sections
  gdb: Prevent wrapping when writing to .segments.tmp
  gdb: If no modules have been loaded, do not try to load module symbols
  gdb: Move runtime module loading into runtime_load_module
  gdb: Conditionally run GDB script logic for dynamically or statically
    positioned GRUB
  gdb: Only connect to remote target once when first sourced
  gdb: Replace module symbol loading implementation with Python one
  gdb: Add functions to make loading from dynamically positioned targets
    easier
  gdb: Add more support for debugging on EFI platforms
  gdb: Allow running user-defined commands at GRUB start
  gdb: Fix issue with breakpoints defined before the GRUB image is
    loaded
  gdb: Add extra early initialization symbols for i386-pc
  gdb: Modify gdb prompt when running gdb_grub script
  docs: Add debugging chapter to development documentation

 config.h.in                 |   3 +
 configure.ac                |  11 ++
 docs/grub-dev.texi          | 233 ++++++++++++++++++++++++++++++++++++
 grub-core/Makefile.core.def |   5 +-
 grub-core/gdb_grub.in       | 159 +++++++++++++++---------
 grub-core/gdb_helper.py.in  | 173 ++++++++++++++++++++++++++
 grub-core/gmodule.pl.in     |  30 -----
 grub-core/kern/efi/debug.c  |  40 +++++++
 grub-core/kern/efi/efi.c    |   4 +-
 grub-core/kern/efi/init.c   |   7 +-
 include/grub/efi/debug.h    |  43 +++++++
 include/grub/efi/efi.h      |   2 +-
 12 files changed, 620 insertions(+), 90 deletions(-)
 create mode 100644 grub-core/gdb_helper.py.in
 delete mode 100644 grub-core/gmodule.pl.in
 create mode 100644 grub-core/kern/efi/debug.c
 create mode 100644 include/grub/efi/debug.h

Range-diff against v4:
 1:  9f273b8fa5 =  1:  9f273b8fa5 gdb: Fix redirection issue in dump_module_sections
 2:  85f68a8369 =  2:  85f68a8369 gdb: Prevent wrapping when writing to .segments.tmp
 3:  88b3973cdb =  3:  88b3973cdb gdb: If no modules have been loaded, do not try to load module symbols
 4:  c0d7da87a8 !  4:  3037c1da91 gdb: Move runtime module loading into runtime_load_module
    @@ Metadata
      ## Commit message ##
         gdb: Move runtime module loading into runtime_load_module
     
    +    By moving this code into a function, it can be run re-utilized while gdb is
    +    running, not just when loading the script. This will also be useful in
    +    some following changes which will make a separate script path for targets
    +    which statically vs dynamically position GRUB code.
    +
      ## grub-core/gdb_grub.in ##
     @@ grub-core/gdb_grub.in: document load_all_modules
      	Load debugging information for all loaded modules.
 5:  4712465374 <  -:  ---------- gdb: Reliably load modules in runtime_load_module
 6:  283021b7b9 <  -:  ---------- gdb: Add functions to make loading from dynamically positioned targets easier
 7:  8f4b7c3bbd <  -:  ---------- gdb: Remove Perl dependency for GRUB GDB script
 8:  055e968779 <  -:  ---------- gdb: If enabled, print line used to load EFI kernel symbols when using gdb_grub script
 9:  64eccfc37e =  5:  f6288016f6 gdb: Conditionally run GDB script logic for dynamically or statically positioned GRUB
10:  5064458dfd =  6:  da13fbe653 gdb: Only connect to remote target once when first sourced
11:  c33e8f57b4 <  -:  ---------- gdb: Allow user defined "onload_<modname>" command to be run when module is loaded
 -:  ---------- >  7:  8e6059955a gdb: Replace module symbol loading implementation with Python one
 -:  ---------- >  8:  878900d69b gdb: Add functions to make loading from dynamically positioned targets easier
 -:  ---------- >  9:  036549604d gdb: Add more support for debugging on EFI platforms
12:  f8a26f3a56 ! 10:  0000959b2f gdb: Allow running user-defined commands at GRUB start
    @@ Metadata
      ## Commit message ##
         gdb: Allow running user-defined commands at GRUB start
     
    -    A new command, run_on_start, is created which handles some complexities
    -    of the EFI platform when breaking on GRUB start. If GRUB start is hooked,
    -    run "onstart" command if it is defned.
    +    A new command, run_on_start, for things to do when just before GRUB starts
    +    executing. Currently, this is setting up the loading of module symbols as
    +    they are loaded and allowing user-defined script to be run if a command
    +    named "onstart" exists. A thbreak, temporary hardware breakpoint, is used
    +    because a software breakpoint would be overwritten when the firmware loads
    +    the GRUB image into memory. And it should be temporary in order to have as
    +    many of the limited hardware breakpoints available to the user as possible.
     
      ## grub-core/gdb_grub.in ##
    -@@ grub-core/gdb_grub.in: end
    +@@ grub-core/gdb_grub.in: source gdb_helper.py
      define dynamic_load_symbols
      	dynamic_load_kernel_exec_symbols $arg0
      
    @@ grub-core/gdb_grub.in: document runtime_load_module
     +define run_on_start
     +	# TODO: Add check to see if _start symbol is defined, if not, then
     +	# the symbols have not yet been loaded and this command will not work.
    -+	watch *_start
    -+	set $break_efi_start_bpnum = $bpnum
    ++	thbreak _start
     +	commands
     +		silent
    -+		delete $break_efi_start_bpnum
    -+		break _start
    -+		commands
    -+			silent
    -+			delete $break_efi_start_bpnum
    -+			set $onstart_name = "onstart"
    -+			is_user_command $onstart_name
    -+			if $ret
    -+				onstart
    -+			end
    -+			continue
    ++
    ++		runtime_load_module
    ++
    ++		if $is_user_command("onstart")
    ++			onstart
     +		end
    -+		set $break_efi_start_bpnum = $bpnum
     +		continue
     +	end
     +end
     +document run_on_start
     +	On some targets, such as x86_64-efi, even if you know where the
    -+	firmware will load the grub image, you can not simply set a break
    ++	firmware will load the GRUB image, you can not simply set a break
     +	point before the image is loaded because loading the image
     +	overwrites the break point in memory. So setup a hardware watch
     +	point, which does not have that problem, and if that gets triggered,
 -:  ---------- > 11:  ac9f52b1d9 gdb: Fix issue with breakpoints defined before the GRUB image is loaded
13:  fbd217a89c ! 12:  eac4405ffb gdb: Add extra early initialization symbols for i386-pc
    @@ Commit message
         gdb: Add extra early initialization symbols for i386-pc
     
         Add symbols for boot.image, disk.image, and lzma_decompress.image if the
    -    target is i386-pc.
    +    target is i386-pc. This is only done for i386-pc because that is the only
    +    target that uses the images. By loading the symbols for these images,
    +    these images can be more easily debugged by allowing the setting of break-
    +    points in that code and to see easily get the value of data symbols.
     
      ## grub-core/gdb_grub.in ##
     @@ grub-core/gdb_grub.in: set confirm off
14:  973f24a485 <  -:  ---------- gdb: Add ability to turn on shell tracing for gdb helper script
 -:  ---------- > 13:  e58715e227 gdb: Modify gdb prompt when running gdb_grub script
15:  d6c6947762 ! 14:  1979dc664e docs: Add debugging chapter to development documentation
    @@ Metadata
      ## Commit message ##
         docs: Add debugging chapter to development documentation
     
    +    Debugging GRUB can be tricky and require arcane knowledge. This will
    +    help those unfamiliar with the process to get started debugging GRUB
    +    with less effort.
    +
      ## docs/grub-dev.texi ##
     @@ docs/grub-dev.texi: This edition documents version @value{VERSION}.
      * Contributing Changes::
    @@ docs/grub-dev.texi: cp minilzo-2.10/*.[hc] grub-core/lib/minilzo
     +does not have the debugging facilities normally provided by an operating
     +system. This chapter aims to provide useful information on some ways to
     +debug GRUB2 for some architectures. It by no means intends to be exhaustive.
    -+The focus will be one X86_64 and i386 architectures. Luckily for some issues
    ++The focus will be one x86_64 and i386 architectures. Luckily for some issues
     +virtual machines have made the ability to debug GRUB2 much easier, and this
     +chapter will focus debugging via the QEMU virtual machine. We will not be
     +going over debugging of the userland tools (eg. grub-install), there are
    @@ docs/grub-dev.texi: cp minilzo-2.10/*.[hc] grub-core/lib/minilzo
     +
     +@example
     +qemu-system-i386 -drive file=disk.img,format=raw \
    -+    -device virtio-scsi-pci,id=scsi0,num_queues=4 -S -s
    ++    -device virtio-scsi-pci,id=scsi0 -S -s
     +@end example
     +
     +This will start a QEMU instance booting from @file{disk.img}. It will pause
    @@ docs/grub-dev.texi: cp minilzo-2.10/*.[hc] grub-core/lib/minilzo
     +@section x86_64-efi
     +
     +Using GDB to debug GRUB2 for the x86_64-efi target has some similarities with
    -+the i386-pc target. Please read be familiar with the @ref{x86_64-efi} section
    ++the i386-pc target. Please read be familiar with the @ref{i386-pc} section
     +when reading this one. Extra care must be used to run QEMU such that it boots
     +a UEFI firmware. This usually involves either using the @samp{-bios} option
     +with a UEFI firmware blob (eg. @file{OVMF.fd}) or loading the firmware via
    @@ docs/grub-dev.texi: cp minilzo-2.10/*.[hc] grub-core/lib/minilzo
     +Like all EFI implementations, on x86_64-efi the (U)EFI firmware that loads
     +the GRUB2 EFI application determines at runtime where the application will
     +be loaded. This means that we do not know where to tell GDB to load the
    -+symbols for the GRUB2 core until the (U)EFI firmware determines it. There
    ++symbols for the GRUB2 core until the (U)EFI firmware determines it. There are
     +two good ways of figuring this out when running in QEMU: use a @ref{OVMF debug log,
     +debug build of OVMF} and check the debug log or have GRUB2 say where it is
     +loaded when it starts. Neither of these are ideal because they both
    @@ docs/grub-dev.texi: cp minilzo-2.10/*.[hc] grub-core/lib/minilzo
     +between QEMU runs. There are exceptions to this, namely that different
     +GRUB2 EFI Applications can be run at different addresses. Also, its been
     +observed that after running the EFI application for the first time, the
    -+second run will many times have a different load address, but subsequent
    ++second run will some times have a different load address, but subsequent
     +runs of the same EFI application will have the same load address as the
    -+second run. This predictability allows us to asume the load address on
    -+subsequent runs and thus load the symbols before GRUB2 starts. The following
    -+command illustrates this, assuming that QEMU is running and waiting for
    -+a debugger connection and the current working directory is where
    -+@file{gdb_grub} resides:
    ++second run. And its a near certainty that if the GRUB EFI binary has changed,
    ++eg. been recompiled, the load address will also be different.
    ++
    ++This ability to predict what the load address will be allows one to assume
    ++the load address on subsequent runs and thus load the symbols before GRUB2
    ++starts. The following command illustrates this, assuming that QEMU is
    ++running and waiting for a debugger connection and the current working
    ++directory is where @file{gdb_grub} resides:
     +
     +@example
    -+gdb -x gdb_grub -ex 'dynamic_load_symbols @var{load address}'
    ++gdb -x gdb_grub -ex 'dynamic_load_symbols @var{address of .text section}'
     +@end example
     +
     +If you load the symbols in this manner and, after continuing execution, do
     +not see output showing the loading of modules symbol, then its very likely
     +that the load address was incorrect.
     +
    ++Another thing to be aware of is how the loading of the GRUB image by the
    ++firmware affects previously set software breakpoints. On x86 platforms,
    ++software breakpoints are implemented by GDB by writing a special processor
    ++instruction at the location of the desired breakpoint. This special instruction
    ++when executed will stop the program execution and hand control to the
    ++debugger, GDB. GDB will first saves the instruction bytes that will be
    ++overwritten at the breakpoint, and will put them back when the breakpoint
    ++is hit. If GRUB is being run for the first time in QEMU, the firmware will
    ++be loading the GRUB image into memory where every byte is already set to 0.
    ++This means that if a breakpoint is set before GRUB is loaded, GDB will save
    ++the 0-byte(s) where the the special instruction will go. Then when the firmware
    ++loads the GRUB image and because it is unaware of the debugger, it will
    ++write the GRUB image to memory, overwriting anything that was there previously,
    ++notably in this case the instruction that implements the software breakpoint.
    ++This will be confusing for the person using GDB because GDB will show the
    ++breakpoint as set, but the brekapoint will never be hit. Furthermore, GDB
    ++then become confused, such that even deleting an recreating the breakpoint
    ++will not create usable breakpoints. The @file{gdb_grub} script takes care of
    ++this by saving the breakpoints just before they are overwritten, and then
    ++restores them at the start of GRUB execution. So breakpoints for GRUB can be
    ++set before GRUB is loaded, but be mindful of this effect if you are confused
    ++as to why breakpoints are not getting hit.
    ++
    ++Also note, that hardware breakpoints do not suffer this problem. They are
    ++implemented by having the breakpoint address in special debug registers on
    ++the CPU. So they can always be set freely without regard to whether GRUB has
    ++been loaded or not. The reason that hardware breakpoints aren't always used
    ++is because there are a limited number of them, usually around 4 on various
    ++CPUs, and specifically exactly 4 for x86 CPUs. The @file{gdb_grub} script
    ++goes out of its way to not use hardware breakpoints internally and when
    ++needed use them as short a time as possible, thus allowing the user to have a
    ++maximal number at their disposal.
    ++
     +
     +@node OVMF debug log
     +@subsection OVMF debug log
    @@ docs/grub-dev.texi: cp minilzo-2.10/*.[hc] grub-core/lib/minilzo
     +@example
     +qemu-system-x86_64 -bios /path/to/debug/OVMF.fd \
     +    -drive file=disk.img,format=raw \
    -+    -device virtio-scsi-pci,id=scsi0,num_queues=4 \
    ++    -device virtio-scsi-pci,id=scsi0 \
     +    -debugcon file:debug.log -global isa-debugcon.iobase=0x402
     +@end example
     +
     +If GRUB2 was started by the (U)EFI firmware, then in the @file{debug.log}
     +file one of the last lines should be a log message like:
    -+@code{Loading driver at 0x00006AEE000 EntryPoint=0x00006AEE756}. This
    -+means that the GRUB2 EFI application was loaded at @code{0x00006AEE000} and
    -+its .text section is at @code{0x00006AEE756}.
    ++@samp{Loading driver at 0x00006AEE000 EntryPoint=0x00006AEE756}. This
    ++means that the GRUB2 EFI application was loaded at @samp{0x00006AEE000} and
    ++its .text section is at @samp{0x00006AEE756}.
     +
     +@node Build GRUB2 to print out the load address
     +@subsection Build GRUB2 to print out the load address
     +
     +GRUB2 can be specially built to output the address of its .text section in
    -+memory by defining @code{PRINT_GDB_SYM_LOAD_CMD} to @code{1} in @file{config.h.in}
    -+before running @command{configure}. The benefit of this method is that it
    -+will work on non-virtualized hardware where the (U)EFI firmware may not
    -+be modifiable.
    ++memory by using the @samp{--enable-efi-debug} configure option. The benefit
    ++of this method is that it will work on non-virtualized hardware where the
    ++(U)EFI firmware may not be modifiable. This option has no effect when booting
    ++with Secure Boot enabled. Otherwise, GRUB will print the gdb command to use
    ++very early in GRUB startup. The text quickly gets overwritten, perhaps even
    ++too quickly to see when booting with a physical monitor as the only output
    ++source. For this reason, a command named "gdbinfo" is enabled which will
    ++print the same output. So the user can get this info at anytime. If GRUB is
    ++crashing before the commandline can be reached, this will be of no help
    ++unfortunately.
     +
      @node Porting
      @chapter Porting
-- 
2.34.1

[PATCH v5 01/14] gdb: Fix redirection issue in dump_module_sections

[Glenn Washburn]

An error in any GDB command causes it to immediately abort with an error,
this includes any command that calls that command. This leads to an issue
in dump_module_sections where an error causes the command to exit without
turning off file redirection. The user then ends up with a GDB command
line where commands output nothing to the console.

Instead do the work of dump_module_sections in the command
dump_module_sections_helper and run the command using GDB's pipe command
which does the redirection and undoes the redirection when it finishes
regardless of any errors in the command.

Also, remove .segments.tmp file prior to loading modules in case one was
left from a previous run.

Signed-off-by: Glenn Washburn <development@efficientek.com>
---
 grub-core/gdb_grub.in | 15 +++++----------
 1 file changed, 5 insertions(+), 10 deletions(-)

diff --git a/grub-core/gdb_grub.in b/grub-core/gdb_grub.in
index e322d3dc10..4e45ad5622 100644
--- a/grub-core/gdb_grub.in
+++ b/grub-core/gdb_grub.in
@@ -10,15 +10,8 @@
 ###
 
 # Add section numbers and addresses to .segments.tmp
-define dump_module_sections
+define dump_module_sections_helper
 	set $mod = $arg0
-
-	# FIXME: save logging status
-	set logging file .segments.tmp
-	set logging redirect on
-	set logging overwrite off
-	set logging on
-
 	printf "%s", $mod->name
 	set $segment = $mod->segment
 	while ($segment)
@@ -26,9 +19,10 @@ define dump_module_sections
 		set $segment = $segment->next
 	end
 	printf "\n"
+end
 
-	set logging off
-	# FIXME: restore logging status
+define dump_module_sections
+	pipe dump_module_sections_helper $arg0 | sh -c 'cat >>.segments.tmp'
 end
 document dump_module_sections
 	Gather information about module whose mod structure was
@@ -59,6 +53,7 @@ document load_module
 end
 
 define load_all_modules
+	shell rm -f .segments.tmp
 	set $this = grub_dl_head
 	while ($this != 0)
 		dump_module_sections $this
-- 
2.34.1

[PATCH v5 02/14] gdb: Prevent wrapping when writing to .segments.tmp

[Glenn Washburn]

GDB logging is redirected to write .segments.tmp, which means that GDB
will wrap lines longer than what it thinks is the screen width
(typically 80 characters). When wrapping does occur it causes gmodule.pl
to misbehave. So disable line wrapping by using GDB's "with" command so
that its guaranteed to return the width to the previous value upon
command completion.

Also disable command tracing when dumping the module sections because
that output will go to .segments.tmp and thus cause gmodule.pl to
misbehave.

Signed-off-by: Glenn Washburn <development@efficientek.com>
---
 grub-core/gdb_grub.in | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/grub-core/gdb_grub.in b/grub-core/gdb_grub.in
index 4e45ad5622..edb5a8872c 100644
--- a/grub-core/gdb_grub.in
+++ b/grub-core/gdb_grub.in
@@ -22,6 +22,10 @@ define dump_module_sections_helper
 end
 
 define dump_module_sections
+	# Set unlimited width so that lines don't get wrapped writing
+	# to .segments.tmp
+	with width 0 -- \
+	with trace-commands off -- \
 	pipe dump_module_sections_helper $arg0 | sh -c 'cat >>.segments.tmp'
 end
 document dump_module_sections
-- 
2.34.1

[PATCH v5 03/14] gdb: If no modules have been loaded, do not try to load module symbols

[Glenn Washburn]

This prevents load_all_modules from failing when called before any
modules have been loaded. Failures in GDB user-defined functions cause
any function which called them to also fail.

Signed-off-by: Glenn Washburn <development@efficientek.com>
---
 grub-core/gdb_grub.in | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/grub-core/gdb_grub.in b/grub-core/gdb_grub.in
index edb5a8872c..fc17e3d899 100644
--- a/grub-core/gdb_grub.in
+++ b/grub-core/gdb_grub.in
@@ -63,7 +63,9 @@ define load_all_modules
 		dump_module_sections $this
 		set $this = $this->next
 	end
-	match_and_load_symbols
+	if (grub_dl_head != 0)
+		match_and_load_symbols
+	end
 end
 document load_all_modules
 	Load debugging information for all loaded modules.
-- 
2.34.1

[PATCH v5 04/14] gdb: Move runtime module loading into runtime_load_module

[Glenn Washburn]

By moving this code into a function, it can be run re-utilized while gdb is
running, not just when loading the script. This will also be useful in
some following changes which will make a separate script path for targets
which statically vs dynamically position GRUB code.

Signed-off-by: Glenn Washburn <development@efficientek.com>
---
 grub-core/gdb_grub.in | 20 +++++++++++++-------
 1 file changed, 13 insertions(+), 7 deletions(-)

diff --git a/grub-core/gdb_grub.in b/grub-core/gdb_grub.in
index fc17e3d899..d525a5a11f 100644
--- a/grub-core/gdb_grub.in
+++ b/grub-core/gdb_grub.in
@@ -71,16 +71,22 @@ document load_all_modules
 	Load debugging information for all loaded modules.
 end
 
+define runtime_load_module
+	break grub_dl_add
+	commands
+		silent
+		load_module mod
+		cont
+	end
+end
+document runtime_load_module
+	Load module symbols at runtime as they are loaded.
+end
+
 ###
 
 set confirm off
 file kernel.exec
 target remote :1234
 
-# inform when module is loaded
-break grub_dl_add
-commands
-	silent
-	load_module mod
-	cont
-end
+runtime_load_module
-- 
2.34.1

[PATCH v5 05/14] gdb: Conditionally run GDB script logic for dynamically or statically positioned GRUB

[Glenn Washburn]

There are broadly two classes of targets to consider when loading symbols
for GRUB, targets that determine where to load GRUB at runtime
(dynamically positioned) and those that do not (statically positioned).
For statically poisitioned targets, symbol loading is determined at link
time, so nothing more needs to be known to load the symbols. For
dynamically positioned targets, such as EFI targets, at runtime symbols
should be offset by an amount that depends on where the runtime chose to
load GRUB.

It is important to not load symbols statically for dynamic targets
because then when subsequently loading the symbols correctly one must
take care to remove the existing static symbols, otherwise there will be
two sets of symbols and GDB seems to prefer the ones loaded first (ie the
static ones).

Use autoconf variables to generate a gdb_grub for a particular target,
which conditionally run startup code depending on if the target uses
static or dynamic loading.

Signed-off-by: Glenn Washburn <development@efficientek.com>
---
 grub-core/gdb_grub.in | 19 ++++++++++++++++---
 1 file changed, 16 insertions(+), 3 deletions(-)

diff --git a/grub-core/gdb_grub.in b/grub-core/gdb_grub.in
index d525a5a11f..620d1def72 100644
--- a/grub-core/gdb_grub.in
+++ b/grub-core/gdb_grub.in
@@ -86,7 +86,20 @@ end
 ###
 
 set confirm off
-file kernel.exec
-target remote :1234
 
-runtime_load_module
+# Note: On EFI and other platforms that load GRUB to an address that is
+# determined at runtime, the symbols in kernel.exec will be wrong.
+# However, we must start by loading some executable file or GDB will
+# fail.
+
+set $platform_efi = $_streq("@platform@", "efi")
+
+if $platform_efi
+	# Only load the executable file, not the symbols
+	exec-file kernel.exec
+else
+	file kernel.exec
+	runtime_load_module
+end
+
+target remote :1234
-- 
2.34.1

[PATCH v5 06/14] gdb: Only connect to remote target once when first sourced

[Glenn Washburn]

The gdb_grub script was originally meant to be run once when GDB first
starts up via the -x argument. So it runs commands unconditionally
assuming that the script has not been run before. Its nice to be able
to source the script again when developing the script to modify/add
commands. So only run the commands not defined in user-defined commands,
if a variable $runonce has already been set and when those commands have
been run to set $runonce.

Signed-off-by: Glenn Washburn <development@efficientek.com>
---
 grub-core/gdb_grub.in | 19 +++++++++++--------
 1 file changed, 11 insertions(+), 8 deletions(-)

diff --git a/grub-core/gdb_grub.in b/grub-core/gdb_grub.in
index 620d1def72..4ca939f69a 100644
--- a/grub-core/gdb_grub.in
+++ b/grub-core/gdb_grub.in
@@ -94,12 +94,15 @@ set confirm off
 
 set $platform_efi = $_streq("@platform@", "efi")
 
-if $platform_efi
-	# Only load the executable file, not the symbols
-	exec-file kernel.exec
-else
-	file kernel.exec
-	runtime_load_module
-end
+if ! $runonce
+	if $platform_efi
+		# Only load the executable file, not the symbols
+		exec-file kernel.exec
+	else
+		file kernel.exec
+		runtime_load_module
+	end
 
-target remote :1234
+	target remote :1234
+	set $runonce = 1
+end
-- 
2.34.1

[PATCH v5 07/14] gdb: Replace module symbol loading implementation with Python one

[Glenn Washburn]

Remove gmodule.pl and rewrite as a python in gdb_helper.py. This removes
PERL dependency for the GRUB GDB script, but adds Python as a dependency.
This is more desirable because Python is tightly integrated with GDB and
can do things not even available to GDB native scripting language. GDB must
be built with Python, however this is not a major limitation because every
major distro non-end-of-life versions build GDB with Python support. And GDB
has had support for Python since around 7.1-ish, which is about a decade.

This re-implementation has an added feature. If there is a user defined
command named "onload_<module name>", then that command will be executed
after the symbols for the specified module are loaded. When debugging a
module it can be desirable to set break points on code in the module.
This is difficult in GRUB because, at GDB start, the module is not loaded
and on EFI platforms its not known ahead of time where the module will
be loaded. So allow users to create an "onload_<modname>" command which
will be run when the module with name "modname" is loaded.

Another addition is a new convenience function is defined
$is_user_command(), which returns true if its string argument is
the name of a user-defined command.

A secondary benefit of these changes is that the script does not write
temporary files and has better error handling capabilities.

Signed-off-by: Glenn Washburn <development@efficientek.com>
---
 grub-core/Makefile.core.def |  4 +-
 grub-core/gdb_grub.in       | 54 ++----------------------
 grub-core/gdb_helper.py.in  | 82 +++++++++++++++++++++++++++++++++++++
 grub-core/gmodule.pl.in     | 30 --------------
 4 files changed, 87 insertions(+), 83 deletions(-)
 create mode 100644 grub-core/gdb_helper.py.in
 delete mode 100644 grub-core/gmodule.pl.in

diff --git a/grub-core/Makefile.core.def b/grub-core/Makefile.core.def
index 95942fc8c9..4951b049ec 100644
--- a/grub-core/Makefile.core.def
+++ b/grub-core/Makefile.core.def
@@ -20,8 +20,8 @@ transform_data = {
 
 transform_data = {
   installdir = platform;
-  name = gmodule.pl;
-  common = gmodule.pl.in;
+  name = gdb_helper.py;
+  common = gdb_helper.py.in;
 };
 
 transform_data = {
diff --git a/grub-core/gdb_grub.in b/grub-core/gdb_grub.in
index 4ca939f69a..fc201204de 100644
--- a/grub-core/gdb_grub.in
+++ b/grub-core/gdb_grub.in
@@ -1,6 +1,6 @@
 ###
 ### Load debuging information about GNU GRUB 2 modules into GDB
-### automatically. Needs readelf, Perl and gmodule.pl script
+### automatically. Needs readelf, Python and gdb_helper.py script
 ###
 ### Has to be launched from the writable and trusted
 ### directory containing *.image and *.module
@@ -9,63 +9,15 @@
 ### Lubomir Kundrak <lkudrak@skosi.org>
 ###
 
-# Add section numbers and addresses to .segments.tmp
-define dump_module_sections_helper
-	set $mod = $arg0
-	printf "%s", $mod->name
-	set $segment = $mod->segment
-	while ($segment)
-		printf " %i 0x%lx", $segment->section, $segment->addr
-		set $segment = $segment->next
-	end
-	printf "\n"
-end
+source gdb_helper.py
 
-define dump_module_sections
-	# Set unlimited width so that lines don't get wrapped writing
-	# to .segments.tmp
-	with width 0 -- \
-	with trace-commands off -- \
-	pipe dump_module_sections_helper $arg0 | sh -c 'cat >>.segments.tmp'
-end
-document dump_module_sections
-	Gather information about module whose mod structure was
-	given for use with match_and_load_symbols
-end
-
-# Generate and execute GDB commands and delete temporary files
-# afterwards
-define match_and_load_symbols
-	shell perl gmodule.pl <.segments.tmp >.loadsym.gdb
-	source .loadsym.gdb
-	shell rm -f .segments.tmp .loadsym.gdb
-end
-document match_and_load_symbols
-	Launch script, that matches section names with information
-	generated by dump_module_sections and load debugging info
-	apropriately
-end
-
-###
-
-define load_module
-	dump_module_sections $arg0
-	match_and_load_symbols
-end
-document load_module
-	Load debugging information for module given as argument.
-end
 
 define load_all_modules
-	shell rm -f .segments.tmp
 	set $this = grub_dl_head
 	while ($this != 0)
-		dump_module_sections $this
+		load_module $this
 		set $this = $this->next
 	end
-	if (grub_dl_head != 0)
-		match_and_load_symbols
-	end
 end
 document load_all_modules
 	Load debugging information for all loaded modules.
diff --git a/grub-core/gdb_helper.py.in b/grub-core/gdb_helper.py.in
new file mode 100644
index 0000000000..4306ef448a
--- /dev/null
+++ b/grub-core/gdb_helper.py.in
@@ -0,0 +1,82 @@
+import os
+import re
+import subprocess
+
+##### Convenience functions #####
+
+class IsUserCommand (gdb.Function):
+  """Set the second argument to true value if first argument is the name
+of a user-defined command.
+"""
+
+  def __init__ (self):
+    super (IsUserCommand, self).__init__ ("is_user_command")
+
+  def invoke (self, fmt, *args):
+    name = fmt.string () % tuple(a.string () for a in args)
+    for line in gdb.execute ("help user-defined", to_string=True).splitlines ():
+      line_parts = line.split(' -- ', 1)
+      if len (line_parts) > 1 and line_parts[0] == name:
+        return True
+    return False
+
+is_user_command = IsUserCommand ()
+
+##### Commands #####
+
+class GrubLoadModuleSymbols (gdb.Command):
+  """Load module symbols at correct locations.
+Takes one argument which is a pointer to a grub_dl_t struct."""
+
+  def __init__ (self):
+    super (GrubLoadModuleSymbols, self).__init__ ("load_module",
+						  gdb.COMMAND_USER,
+						  gdb.COMPLETE_EXPRESSION)
+
+  def invoke (self, arg, from_tty):
+    self.dont_repeat ()
+    args = gdb.string_to_argv (arg)
+    self.mod = gdb.parse_and_eval (args[0])
+    sections = self.get_section_offsets ()
+    section_names = self.get_section_names ()
+
+    sym_load_cmd_parts = ["add-symbol-file",
+			  "%s.module" % (self.mod['name'].string (),)]
+    for idx, addr in sections:
+      section_name = section_names[idx]
+      if section_name == ".text":
+        sym_load_cmd_parts.append (addr)
+      else:
+        sym_load_cmd_parts.extend (["-s", section_name, addr])
+    gdb.execute (' '.join (sym_load_cmd_parts))
+
+    if is_user_command.invoke (gdb.Value ("onload_%s"), self.mod['name']):
+      gdb.execute ("onload_%s (grub_dl_t)%s" % (self.mod['name'].string (),
+						self.mod.format_string (format='x')))
+
+  def get_section_offsets (self):
+    sections = []
+    segment = self.mod['segment']
+    while segment:
+      sections.append ((int (segment['section']), segment['addr'].format_string (format='x')))
+      segment = segment['next']
+    return sections
+
+  def get_section_names (self):
+    re_index = re.compile ("^\s+\[\s*(\d+)\] (\S*)")
+    names = {}
+    modfilename = "%s.mod" % (self.mod['name'].string (),)
+
+    if not os.path.exists (modfilename):
+      raise RuntimeError ("%s not found in current directory" % (modfilename,))
+
+    c = subprocess.run (["readelf", "-SW", modfilename], text=True, capture_output=True)
+    for line in c.stdout.splitlines ()[4:]:
+      m = re_index.match (line)
+      if not m:
+        continue
+      idx, name = m.groups ()
+      names[int (idx)] = name
+    return names
+
+grub_load_module = GrubLoadModuleSymbols ()
diff --git a/grub-core/gmodule.pl.in b/grub-core/gmodule.pl.in
deleted file mode 100644
index 78aa1e64eb..0000000000
--- a/grub-core/gmodule.pl.in
+++ /dev/null
@@ -1,30 +0,0 @@
-###
-### Generate GDB commands, that load symbols for specified module,
-### with proper section relocations. See .gdbinit
-###
-### $Id: gmodule.pl,v 1.2 2006/05/14 11:38:42 lkundrak Exp lkundrak $
-### Lubomir Kundrak <lkudrak@skosi.org>
-###
-
-use strict;
-
-while (<>) {
-	my ($name, %sections) = split;
-
-	print "add-symbol-file $name.module";
-
-	open (READELF, "readelf -S $name.mod |") or die;
-	while (<READELF>) {
-		/\[\s*(\d+)\]\s+(\.\S+)/ or next;
-
-		if ($2 eq '.text') {
-			print " $sections{$1}";
-			next;
-		}
-
-		print " -s $2 $sections{$1}"
-			if ($sections{$1} ne '0x0' and $sections{$1} ne '');
-	};
-	close (READELF);
-	print "\n";
-}
-- 
2.34.1

[PATCH v5 08/14] gdb: Add functions to make loading from dynamically positioned targets easier

[Glenn Washburn]

Many targets, such as EFI, load GRUB at addresses that are determined at
runtime. So the load addresses in kernel.exec will almost certainly be
wrong. Given the address of the start of the text segment, these
functions will tell GDB to load the symbols at the proper locations. It
is left up to the user to determine how to get the text address of the
loaded GRUB image.

Signed-off-by: Glenn Washburn <development@efficientek.com>
---
 grub-core/gdb_grub.in      | 21 ++++++++-
 grub-core/gdb_helper.py.in | 87 ++++++++++++++++++++++++++++++++++++++
 2 files changed, 107 insertions(+), 1 deletion(-)

diff --git a/grub-core/gdb_grub.in b/grub-core/gdb_grub.in
index fc201204de..18ce6b0eb2 100644
--- a/grub-core/gdb_grub.in
+++ b/grub-core/gdb_grub.in
@@ -1,6 +1,6 @@
 ###
 ### Load debuging information about GNU GRUB 2 modules into GDB
-### automatically. Needs readelf, Python and gdb_helper.py script
+### automatically. Needs readelf, objdump, Python and gdb_helper.py script
 ###
 ### Has to be launched from the writable and trusted
 ### directory containing *.image and *.module
@@ -12,6 +12,25 @@
 source gdb_helper.py
 
 
+define dynamic_load_symbols
+	dynamic_load_kernel_exec_symbols $arg0
+
+	# We may have been very late to loading the kernel.exec symbols and
+	# and modules may already be loaded. So load symbols for any already
+	# loaded.
+	load_all_modules
+
+	if $is_grub_loaded()
+		runtime_load_module
+	end
+end
+document dynamic_load_symbols
+	Load debugging symbols from kernel.exec and any loaded modules given
+	the address of the .text segment of the UEFI binary in memory. Also
+	setup session to automatically load module symbols for modules loaded
+	in the future.
+end
+
 define load_all_modules
 	set $this = grub_dl_head
 	while ($this != 0)
diff --git a/grub-core/gdb_helper.py.in b/grub-core/gdb_helper.py.in
index 4306ef448a..8d5ee1d292 100644
--- a/grub-core/gdb_helper.py.in
+++ b/grub-core/gdb_helper.py.in
@@ -4,6 +4,23 @@ import subprocess
 
 ##### Convenience functions #####
 
+class IsGrubLoaded (gdb.Function):
+  """Return 1 if GRUB has been loaded in memory, otherwise 0.
+The hueristic used is checking if the first 4 bytes of the memory pointed
+to by the _start symbol are not 0. This is true for QEMU on the first run
+of GRUB. This may not be true on physical hardware, where memory is not
+necessarily cleared on soft reset. This may not also be true in QEMU on
+soft resets. Also this many not be true when chainloading GRUB.
+"""
+
+  def __init__ (self):
+    super (IsGrubLoaded, self).__init__ ("is_grub_loaded")
+
+  def invoke (self):
+    return int (gdb.parse_and_eval ("*(int *) _start")) != 0
+
+is_grub_loaded = IsGrubLoaded ()
+
 class IsUserCommand (gdb.Function):
   """Set the second argument to true value if first argument is the name
 of a user-defined command.
@@ -24,6 +41,76 @@ is_user_command = IsUserCommand ()
 
 ##### Commands #####
 
+# Loading symbols is complicated by the fact that kernel.exec is an ELF
+# ELF binary, but the UEFI runtime is PE32+. All the data sections of
+# the ELF binary are concatenated (accounting for ELF section alignment)
+# and put into one .data section of the PE32+ runtime image. So given
+# the load address of the .data PE32+ section we can determine the
+# addresses each ELF data section maps to. The UEFI application is
+# loaded into memory just as it is laid out in the file. It is not
+# assumed that the binary is available, but it is known that the .text
+# section directly precedes the .data section and that .data is EFI
+# page aligned. Using this, the .data offset can be found from the .text
+# address.
+class GrubLoadKernelExecSymbols (gdb.Command):
+  """Load debugging symbols from kernel.exec given the address of the
+.text segment of the UEFI binary in memory."""
+
+  PE_SECTION_ALIGN = 12
+
+  def __init__ (self):
+    super (GrubLoadKernelExecSymbols, self).__init__ ("dynamic_load_kernel_exec_symbols",
+						      gdb.COMMAND_USER,
+						      gdb.COMPLETE_EXPRESSION)
+
+  def invoke (self, arg, from_tty):
+    self.dont_repeat ()
+    args = gdb.string_to_argv (arg)
+
+    if len (args) != 1:
+      raise RuntimeError ("dynamic_load_kernel_exec_symbols expects exactly one argument")
+
+    sections = self.parse_objdump_sections ("kernel.exec")
+    pe_text = args[0]
+    text_size = [s['size'] for s in sections if s['name'] == '.text'][0]
+    pe_data_offset = self.alignup (text_size, self.PE_SECTION_ALIGN)
+
+    sym_load_cmd_parts = ["add-symbol-file", "kernel.exec", pe_text]
+    offset = 0
+    for section in sections:
+      if 'DATA' in section["flags"] or section["name"] == ".bss":
+        offset = self.alignup (offset, section["align"])
+        sym_load_cmd_parts.extend (["-s", section["name"], "(%s+0x%x+0x%x)" % (pe_text, pe_data_offset, offset)])
+        offset += section["size"]
+    gdb.execute (' '.join (sym_load_cmd_parts))
+
+  @staticmethod
+  def parse_objdump_sections (filename):
+    fields = ("idx", "name", "size", "vma", "lma", "fileoff", "align")
+    re_section = re.compile ("^\s*" + "\s+".join(["(?P<%s>\S+)" % f for f in fields]))
+    c = subprocess.run (["objdump", "-h", filename], text=True, capture_output=True)
+    section_lines = c.stdout.splitlines ()[5:]
+    sections = []
+
+    for i in range (len (section_lines) >> 1):
+      m = re_section.match (section_lines[i * 2])
+      s = dict (m.groupdict ())
+      for f in ("size", "vma", "lma", "fileoff"):
+        s[f] = int (s[f], 16)
+      s["idx"] = int (s["idx"])
+      s["align"] = int (s["align"].split ("**", 1)[1])
+      s["flags"] = section_lines[(i * 2) + 1].strip ().split (", ")
+      sections.append (s)
+    return sections
+
+  @staticmethod
+  def alignup (addr, align):
+    pad = (addr % (1 << align)) and 1 or 0
+    return ((addr >> align) + pad) << align
+
+dynamic_load_kernel_exec_symbols = GrubLoadKernelExecSymbols ()
+
+
 class GrubLoadModuleSymbols (gdb.Command):
   """Load module symbols at correct locations.
 Takes one argument which is a pointer to a grub_dl_t struct."""
-- 
2.34.1

[PATCH v5 09/14] gdb: Add more support for debugging on EFI platforms

[Glenn Washburn]

If the configure option --enable-efi-debug is given, then enable the
printing early in EFI startup of the command needed to load symbols for
the GRUB EFI kernel. This is needed because EFI firmware determines where
to load the GRUB EFI at runtime, and so the relevant addresses are not
known ahead of time. This is not printed when secure boot is enabled.

The command is a custom command defined in the gdb_grub GDB script. So
GDB should be started with the script as an argument to the -x option or
sourced into an active GDB session before running the outputted command.

Also a command named "gdbinfo" is enabled which allows the user to print
the gdb command string on-demand, which can be valuable as the printing
early in EFI startup is quickly replaced by other text. So if using a
physical screen it may appear too briefly to be registered.

Co-developed-by: Peter Jones <pjones@redhat.com>
Signed-off-by: Glenn Washburn <development@efficientek.com>
---
 config.h.in                 |  3 +++
 configure.ac                | 11 ++++++++++
 grub-core/Makefile.core.def |  1 +
 grub-core/kern/efi/debug.c  | 40 ++++++++++++++++++++++++++++++++++
 grub-core/kern/efi/efi.c    |  4 ++--
 grub-core/kern/efi/init.c   |  7 +++++-
 include/grub/efi/debug.h    | 43 +++++++++++++++++++++++++++++++++++++
 include/grub/efi/efi.h      |  2 +-
 8 files changed, 107 insertions(+), 4 deletions(-)
 create mode 100644 grub-core/kern/efi/debug.c
 create mode 100644 include/grub/efi/debug.h

diff --git a/config.h.in b/config.h.in
index 4d1e50eba7..9f88be1593 100644
--- a/config.h.in
+++ b/config.h.in
@@ -13,6 +13,9 @@
 #define MM_DEBUG @MM_DEBUG@
 #endif
 
+/* Define to 1 to enable printing of gdb command to load module symbols.  */
+#define PRINT_GDB_SYM_LOAD_CMD @EFI_DEBUG@
+
 /* Define to 1 to enable disk cache statistics.  */
 #define DISK_CACHE_STATS @DISK_CACHE_STATS@
 #define BOOT_TIME_STATS @BOOT_TIME_STATS@
diff --git a/configure.ac b/configure.ac
index 93626b7982..4bf477b072 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1521,6 +1521,17 @@ else
 fi
 AC_SUBST([MM_DEBUG])
 
+# EFI debugging.
+AC_ARG_ENABLE([efi-debug],
+	      AS_HELP_STRING([--enable-efi-debug],
+                             [include aides for debugging the EFI binary]))
+if test x$enable_efi_debug = xyes; then
+  EFI_DEBUG=1
+else
+  EFI_DEBUG=0
+fi
+AC_SUBST([EFI_DEBUG])
+
 AC_ARG_ENABLE([cache-stats],
 	      AS_HELP_STRING([--enable-cache-stats],
                              [enable disk cache statistics collection]))
diff --git a/grub-core/Makefile.core.def b/grub-core/Makefile.core.def
index 4951b049ec..9891253aac 100644
--- a/grub-core/Makefile.core.def
+++ b/grub-core/Makefile.core.def
@@ -206,6 +206,7 @@ kernel = {
 
   efi = disk/efi/efidisk.c;
   efi = kern/efi/efi.c;
+  efi = kern/efi/debug.c;
   efi = kern/efi/init.c;
   efi = kern/efi/mm.c;
   efi = term/efi/console.c;
diff --git a/grub-core/kern/efi/debug.c b/grub-core/kern/efi/debug.c
new file mode 100644
index 0000000000..c4f5561587
--- /dev/null
+++ b/grub-core/kern/efi/debug.c
@@ -0,0 +1,40 @@
+/*
+ *  GRUB  --  GRand Unified Bootloader
+ *  Copyright (C) 2022  Free Software Foundation, Inc.
+ *
+ *  GRUB is free software: you can redistribute it and/or modify
+ *  it under the terms of the GNU General Public License as published by
+ *  the Free Software Foundation, either version 3 of the License, or
+ *  (at your option) any later version.
+ *
+ *  GRUB is distributed in the hope that it will be useful,
+ *  but WITHOUT ANY WARRANTY; without even the implied warranty of
+ *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ *  GNU General Public License for more details.
+ *
+ *  You should have received a copy of the GNU General Public License
+ *  along with GRUB.  If not, see <http://www.gnu.org/licenses/>.
+ */
+/* debug.c - aides for debugging the EFI application */
+
+#include <grub/efi/debug.h>
+#include <grub/command.h>
+#include <grub/i18n.h>
+
+__attribute__ ((unused)) static grub_err_t
+grub_cmd_gdbinfo (struct grub_command *cmd __attribute__ ((unused)),
+		  int argc __attribute__ ((unused)),
+		  char **args __attribute__ ((unused)))
+{
+  grub_efi_print_gdb_info ();
+  return 0;
+}
+
+void
+grub_efi_register_debug_commands (void)
+{
+#if PRINT_GDB_SYM_LOAD_CMD
+  grub_register_command_lockdown ("gdbinfo", grub_cmd_gdbinfo, 0,
+				  N_("Print infomation useful for GDB debugging"));
+#endif
+}
diff --git a/grub-core/kern/efi/efi.c b/grub-core/kern/efi/efi.c
index cf49d6357e..17bd06f7e6 100644
--- a/grub-core/kern/efi/efi.c
+++ b/grub-core/kern/efi/efi.c
@@ -299,7 +299,7 @@ grub_efi_get_variable (const char *var, const grub_efi_guid_t *guid,
 /* Search the mods section from the PE32/PE32+ image. This code uses
    a PE32 header, but should work with PE32+ as well.  */
 grub_addr_t
-grub_efi_modules_addr (void)
+grub_efi_section_addr (const char *section_name)
 {
   grub_efi_loaded_image_t *image;
   struct grub_msdos_image_header *header;
@@ -328,7 +328,7 @@ grub_efi_modules_addr (void)
        i < coff_header->num_sections;
        i++, section++)
     {
-      if (grub_strcmp (section->name, "mods") == 0)
+      if (grub_strcmp (section->name, section_name) == 0)
 	break;
     }
 
diff --git a/grub-core/kern/efi/init.c b/grub-core/kern/efi/init.c
index b67bc73a1b..edacaddf06 100644
--- a/grub-core/kern/efi/init.c
+++ b/grub-core/kern/efi/init.c
@@ -19,6 +19,7 @@
 
 #include <grub/efi/efi.h>
 #include <grub/efi/console.h>
+#include <grub/efi/debug.h>
 #include <grub/efi/disk.h>
 #include <grub/efi/sb.h>
 #include <grub/lockdown.h>
@@ -104,7 +105,7 @@ grub_addr_t grub_modbase;
 void
 grub_efi_init (void)
 {
-  grub_modbase = grub_efi_modules_addr ();
+  grub_modbase = grub_efi_section_addr ("mods");
   /* First of all, initialize the console so that GRUB can display
      messages.  */
   grub_console_init ();
@@ -123,11 +124,15 @@ grub_efi_init (void)
       grub_lockdown ();
       grub_shim_lock_verifier_setup ();
     }
+  else
+    grub_efi_print_gdb_info ();
 
   efi_call_4 (grub_efi_system_table->boot_services->set_watchdog_timer,
 	      0, 0, 0, NULL);
 
   grub_efidisk_init ();
+
+  grub_efi_register_debug_commands ();
 }
 
 void (*grub_efi_net_config) (grub_efi_handle_t hnd,
diff --git a/include/grub/efi/debug.h b/include/grub/efi/debug.h
new file mode 100644
index 0000000000..dcf0bdbc4d
--- /dev/null
+++ b/include/grub/efi/debug.h
@@ -0,0 +1,43 @@
+/*
+ *  GRUB  --  GRand Unified Bootloader
+ *  Copyright (C) 2022  Free Software Foundation, Inc.
+ *
+ *  GRUB is free software: you can redistribute it and/or modify
+ *  it under the terms of the GNU General Public License as published by
+ *  the Free Software Foundation, either version 3 of the License, or
+ *  (at your option) any later version.
+ *
+ *  GRUB is distributed in the hope that it will be useful,
+ *  but WITHOUT ANY WARRANTY; without even the implied warranty of
+ *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ *  GNU General Public License for more details.
+ *
+ *  You should have received a copy of the GNU General Public License
+ *  along with GRUB.  If not, see <http://www.gnu.org/licenses/>.
+ */
+/* debug.h - declare variables and functions for EFI debugging support */
+
+#ifndef GRUB_EFI_DEBUG_HEADER
+#define GRUB_EFI_DEBUG_HEADER	1
+
+#include <grub/efi/efi.h>
+#include <grub/misc.h>
+
+
+void grub_efi_register_debug_commands (void);
+
+void
+grub_efi_print_gdb_info (void)
+{
+#if PRINT_GDB_SYM_LOAD_CMD
+  grub_addr_t text;
+
+  text = grub_efi_section_addr (".text");
+  if (!text)
+    return;
+
+  grub_printf ("dynamic_load_symbols %p\n", (void *)text);
+#endif
+}
+
+#endif /* ! GRUB_EFI_DEBUG_HEADER */
diff --git a/include/grub/efi/efi.h b/include/grub/efi/efi.h
index e61272de53..586ac856b5 100644
--- a/include/grub/efi/efi.h
+++ b/include/grub/efi/efi.h
@@ -109,7 +109,7 @@ grub_err_t grub_arch_efi_linux_boot_image(grub_addr_t addr, grub_size_t size,
                                            char *args);
 #endif
 
-grub_addr_t grub_efi_modules_addr (void);
+grub_addr_t grub_efi_section_addr (const char *section);
 
 void grub_efi_mm_init (void);
 void grub_efi_mm_fini (void);
-- 
2.34.1

[PATCH v5 10/14] gdb: Allow running user-defined commands at GRUB start

[Glenn Washburn]

A new command, run_on_start, for things to do when just before GRUB starts
executing. Currently, this is setting up the loading of module symbols as
they are loaded and allowing user-defined script to be run if a command
named "onstart" exists. A thbreak, temporary hardware breakpoint, is used
because a software breakpoint would be overwritten when the firmware loads
the GRUB image into memory. And it should be temporary in order to have as
many of the limited hardware breakpoints available to the user as possible.

Signed-off-by: Glenn Washburn <development@efficientek.com>
---
 grub-core/gdb_grub.in | 30 ++++++++++++++++++++++++++++++
 1 file changed, 30 insertions(+)

diff --git a/grub-core/gdb_grub.in b/grub-core/gdb_grub.in
index 18ce6b0eb2..281dfb5927 100644
--- a/grub-core/gdb_grub.in
+++ b/grub-core/gdb_grub.in
@@ -15,6 +15,8 @@ source gdb_helper.py
 define dynamic_load_symbols
 	dynamic_load_kernel_exec_symbols $arg0
 
+	run_on_start
+
 	# We may have been very late to loading the kernel.exec symbols and
 	# and modules may already be loaded. So load symbols for any already
 	# loaded.
@@ -54,6 +56,33 @@ document runtime_load_module
 	Load module symbols at runtime as they are loaded.
 end
 
+define run_on_start
+	# TODO: Add check to see if _start symbol is defined, if not, then
+	# the symbols have not yet been loaded and this command will not work.
+	thbreak _start
+	commands
+		silent
+
+		runtime_load_module
+
+		if $is_user_command("onstart")
+			onstart
+		end
+		continue
+	end
+end
+document run_on_start
+	On some targets, such as x86_64-efi, even if you know where the
+	firmware will load the GRUB image, you can not simply set a break
+	point before the image is loaded because loading the image
+	overwrites the break point in memory. So setup a hardware watch
+	point, which does not have that problem, and if that gets triggered,
+	then reset the break point. If a user-defined command named
+	"onstart" exists it will be run after the start is hit.
+	NOTE: This assumes symbols have already been correctly loaded for
+	the EFI application.
+end
+
 ###
 
 set confirm off
@@ -71,6 +100,7 @@ if ! $runonce
 		exec-file kernel.exec
 	else
 		file kernel.exec
+		run_on_start
 		runtime_load_module
 	end
 
-- 
2.34.1

[PATCH v5 11/14] gdb: Fix issue with breakpoints defined before the GRUB image is loaded

[Glenn Washburn]

On some platforms, notably x86, software breakpoints set in GDB before
the GRUB image is loaded will be cleared when the image is loaded. This
is because the breakpoints work by overwriting the memory of the break-
point location with a special instruction which when hit will cause the
debugger to stop execution. Just before execution is resumed by the
debugger, the original instruction bytes are put back. When a breakpoint
is set before the GRUB image is loaded, the special debugger instruction
will be written to memory and when the GRUB image is loaded by the
firmware, which has no knowledge of the debugger, the debugger instruction
is overwritten. To the GDB user, GDB will show the breakpoint as set, but
it will never be hit. Furthermore, GDB now becomes confused, such that
even deleting and re-setting the breakpoint after the GRUB image is loaded
will not allow for a working breakpoint.

To work around this, in run_on_start, first a watchpoint is set on _start,
which will be triggered when the firmware starts loading the GRUB image.
When the _start watchpoint is hit, the current breakpoints are saved to a
file and then deleted by GDB before they can be overwritten by the firmware
and confuse GDB. Then a temporary software breakpoint is set on _start,
which will get triggered when the firmware hands off to GRUB to execute. In
that breakpoint load the previously saved and deleted breakpoints now that
there is no worry of them getting overwritten by the firmware.

Note that watchpoints are generally types of hardware breakpoints on x86, so
its deleted as soon as it gets triggered so that a minimal set of hardware
breakpoints are used, allowing more for the user.

Signed-off-by: Glenn Washburn <development@efficientek.com>
---
 grub-core/gdb_grub.in | 33 +++++++++++++++++++++++++++------
 1 file changed, 27 insertions(+), 6 deletions(-)

diff --git a/grub-core/gdb_grub.in b/grub-core/gdb_grub.in
index 281dfb5927..8e89bbf368 100644
--- a/grub-core/gdb_grub.in
+++ b/grub-core/gdb_grub.in
@@ -59,14 +59,35 @@ end
 define run_on_start
 	# TODO: Add check to see if _start symbol is defined, if not, then
 	# the symbols have not yet been loaded and this command will not work.
-	thbreak _start
+	watch *_start
+	set $break_efi_start_bpnum = $bpnum
 	commands
 		silent
-
-		runtime_load_module
-
-		if $is_user_command("onstart")
-			onstart
+		delete $break_efi_start_bpnum
+
+		# Save the breakpoints here before the GRUB image is loaded
+		# into memory, then delete them. Later they will be reloaded
+		# once the GRUB image has been loaded. This avoids the issue
+		# where the loading of the GRUB image overwrites the software
+		# breakpoints, thus confusing GDB and effectively clearing
+		# those breakpoints.
+		save breakpoints .early-breakpoints.gdb
+		delete breakpoints
+
+		tbreak _start
+		commands
+			silent
+
+			# Reload the breakpoints now that the GRUB image has
+			# finished being loaded into memory.
+			source .early-breakpoints.gdb
+
+			runtime_load_module
+
+			if $is_user_command("onstart")
+				onstart
+			end
+			continue
 		end
 		continue
 	end
-- 
2.34.1

[PATCH v5 12/14] gdb: Add extra early initialization symbols for i386-pc

[Glenn Washburn]

Add symbols for boot.image, disk.image, and lzma_decompress.image if the
target is i386-pc. This is only done for i386-pc because that is the only
target that uses the images. By loading the symbols for these images,
these images can be more easily debugged by allowing the setting of break-
points in that code and to see easily get the value of data symbols.

Signed-off-by: Glenn Washburn <development@efficientek.com>
---
 grub-core/gdb_grub.in | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/grub-core/gdb_grub.in b/grub-core/gdb_grub.in
index 8e89bbf368..f188a842ab 100644
--- a/grub-core/gdb_grub.in
+++ b/grub-core/gdb_grub.in
@@ -114,12 +114,18 @@ set confirm off
 # fail.
 
 set $platform_efi = $_streq("@platform@", "efi")
+set $target = "@target_cpu@-@platform@"
 
 if ! $runonce
 	if $platform_efi
 		# Only load the executable file, not the symbols
 		exec-file kernel.exec
 	else
+		if $_streq($target, "i386-pc")
+			add-symbol-file boot.image
+			add-symbol-file diskboot.image
+			add-symbol-file lzma_decompress.image
+		end
 		file kernel.exec
 		run_on_start
 		runtime_load_module
-- 
2.34.1

[PATCH v5 13/14] gdb: Modify gdb prompt when running gdb_grub script

[Glenn Washburn]

This will let users know that the GDB session is using the GRUB gdb scripts.

Signed-off-by: Glenn Washburn <development@efficientek.com>
---
 grub-core/gdb_helper.py.in | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/grub-core/gdb_helper.py.in b/grub-core/gdb_helper.py.in
index 8d5ee1d292..5ed9eab0f5 100644
--- a/grub-core/gdb_helper.py.in
+++ b/grub-core/gdb_helper.py.in
@@ -2,6 +2,10 @@ import os
 import re
 import subprocess
 
+def prompt_hook (current_prompt):
+  return "(grub gdb) "
+gdb.prompt_hook = prompt_hook
+
 ##### Convenience functions #####
 
 class IsGrubLoaded (gdb.Function):
-- 
2.34.1

[PATCH v5 14/14] docs: Add debugging chapter to development documentation

[Glenn Washburn]

Debugging GRUB can be tricky and require arcane knowledge. This will
help those unfamiliar with the process to get started debugging GRUB
with less effort.

Signed-off-by: Glenn Washburn <development@efficientek.com>
---
 docs/grub-dev.texi | 233 +++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 233 insertions(+)

diff --git a/docs/grub-dev.texi b/docs/grub-dev.texi
index f76fc658bf..18f09a48e7 100644
--- a/docs/grub-dev.texi
+++ b/docs/grub-dev.texi
@@ -79,6 +79,7 @@ This edition documents version @value{VERSION}.
 * Contributing Changes::
 * Setting up and running test suite::
 * Updating External Code::
+* Debugging::
 * Porting::
 * Error Handling::
 * Stack and heap size::
@@ -595,6 +596,238 @@ cp minilzo-2.10/*.[hc] grub-core/lib/minilzo
 rm -r minilzo-2.10*
 @end example
 
+@node Debugging
+@chapter Debugging
+
+GRUB2 can be difficult to debug because it runs on the bare-metal and thus
+does not have the debugging facilities normally provided by an operating
+system. This chapter aims to provide useful information on some ways to
+debug GRUB2 for some architectures. It by no means intends to be exhaustive.
+The focus will be one x86_64 and i386 architectures. Luckily for some issues
+virtual machines have made the ability to debug GRUB2 much easier, and this
+chapter will focus debugging via the QEMU virtual machine. We will not be
+going over debugging of the userland tools (eg. grub-install), there are
+many tutorials on debugging programs in userland.
+
+You will need GDB and the QEMU binaries for your system, on Debian these
+can be installed with the @samp{gdb} and @samp{qemu-system-x86} packages.
+Also it is assumed that you have already successfully compiled GRUB2 from
+source for the target specified in the section below and have some
+familiarity with GDB. When GRUB2 is built it will create many different
+binaries. The ones of concern will be in the @file{grub-core}
+directory of the GRUB2 build dir. To aide in debugging we will want the
+debugging symbols generated during the build because these symbols are not
+kept in the binaries which get installed to the boot location. The build
+process outputs two sets of binaries, one without symbols which gets executed
+at boot, and another set of ELF images with debugging symbols. The built
+images with debugging symbols will have a @file{.image} suffix, and the ones
+without a @file{.img} suffix. Similarly, loadable modules with debugging
+symbols will have a @file{.module} suffix, and ones without a @file{.mod}
+suffix. In the case of the kernel the binary with symbols is named
+@file{kernel.exec}.
+
+In the following sections, information will be provided on debugging on
+various targets using @command{gdb} and the @samp{gdb_grub} GDB script.
+
+@menu
+* i386-pc::
+* x86_64-efi::
+@end menu
+
+@node i386-pc
+@section i386-pc
+
+The i386-pc target is a good place to start when first debugging GRUB2
+because in some respects its easier than EFI platforms. The reason being
+that the initial load address is always known in advance. To start
+debugging GRUB2 first QEMU must be started in GDB stub mode. The following
+command is a simple illustration:
+
+@example
+qemu-system-i386 -drive file=disk.img,format=raw \
+    -device virtio-scsi-pci,id=scsi0 -S -s
+@end example
+
+This will start a QEMU instance booting from @file{disk.img}. It will pause
+at start waiting for a GDB instance to attach to it. You should change
+@file{disk.img} to something more appropriate. A block device can be used,
+but you may need to run QEMU as a privileged user.
+
+To connect to this QEMU instance with GDB, the @code{target remote} GDB
+command must be used. We also need to load a binary image, preferably with
+symbols. This can be done using the GDB command @code{file kernel.exec}, if
+GDB is started from the @file{grub-core} directory in the GRUB2 build
+directory. GRUB2 developers have made this more simple by including a GDB
+script which does much of the setup. This file at @file{grub-core/gdb_grub}
+of the build directory and is also installed via @command{make install}.
+If not building GRUB, the distribution may have a package which installs
+this GDB script along with debug symbol binaries, such as Debian's
+@samp{grub-pc-dbg} package. The GDB scripts is intended to by used
+like so, assuming:
+
+@example
+cd $(dirname /path/to/script/gdb_grub)
+gdb -x gdb_grub
+@end example
+
+Once GDB has been started with the @file{gdb_grub} script it will
+automatically connect to the QEMU instance. You can then do things you
+normally would in GDB like set a break point on @var{grub_main}.
+
+Setting breakpoints in modules is trickier since they haven't been loaded
+yet and are loaded at addresses determined at runtime. The module could be
+loaded to different addresses in different QEMU instances. The debug symbols
+in the modules @file{.module} binary, thus are always wrong, and GDB needs
+to be told where to load the symbols to. But this must happen at runtime
+after GRUB2 has determined where the module will get loaded. Luckily the
+@file{gdb_grub} script takes care of this with the @command{runtime_load_module}
+command, which configures GDB to watch for GRUB2 module loading and when
+it does add the module symbols with the appropriate offset.
+
+@node x86_64-efi
+@section x86_64-efi
+
+Using GDB to debug GRUB2 for the x86_64-efi target has some similarities with
+the i386-pc target. Please read be familiar with the @ref{i386-pc} section
+when reading this one. Extra care must be used to run QEMU such that it boots
+a UEFI firmware. This usually involves either using the @samp{-bios} option
+with a UEFI firmware blob (eg. @file{OVMF.fd}) or loading the firmware via
+pflash. This document will not go further into how to do this as there are
+ample resource on the web.
+
+Like all EFI implementations, on x86_64-efi the (U)EFI firmware that loads
+the GRUB2 EFI application determines at runtime where the application will
+be loaded. This means that we do not know where to tell GDB to load the
+symbols for the GRUB2 core until the (U)EFI firmware determines it. There are
+two good ways of figuring this out when running in QEMU: use a @ref{OVMF debug log,
+debug build of OVMF} and check the debug log or have GRUB2 say where it is
+loaded when it starts. Neither of these are ideal because they both
+generally give the information after GRUB2 is already running, which makes
+debugging early boot infeasible. Technically, the first method does give
+the load address before GRUB2 is run, but without debugging the EFI firmware
+with symbols, the author currently does not know how to cause the OVMF
+firmware to pause at that point to use the load address before GRUB2 is run.
+
+Even after getting the application load address, the loading of core symbols
+is complicated by the fact that the debugging symbols for the kernel are in
+an ELF binary named @file{kernel.exec} while what is in memory are sections
+for the PE32+ EFI binary. When @command{grub-mkimage} creates the PE32+
+binary it condenses several segments from the ELF kernel binary into one
+.data section in the PE32+ binary. This must be taken into account to
+properly load the other non-text sections. Otherwise, GDB will work as
+expected when breaking on functions, but, for instance, global variables
+will point to the wrong address in memory and thus give incorrect values
+(which can be difficult to debug).
+
+The calculating of the correct offsets for sections when loading symbol
+files are taken care of when loading the kernel symbols via the user-defined
+GDB command @command{dynamic_load_kernel_exec_symbols}, which takes one
+argument, the address where the text section is loaded, as determined by
+one of the methods above. Alternatively, the command @command{dynamic_load_symbols}
+with the text section address as an agrument can be called to load the
+kernel symbols and setup loading the module symbols as they are loaded at
+runtime.
+
+In the author's experience, when debugging with QEMU and OVMF, to have
+debugging symbols loaded at the start of GRUB2 execution the GRUB2 EFI
+application must be run via QEMU at least once prior in order to get the
+load address. Two methods for obtaining the load address are described in
+two subsections below. Generally speaking, the load address does not change
+between QEMU runs. There are exceptions to this, namely that different
+GRUB2 EFI Applications can be run at different addresses. Also, its been
+observed that after running the EFI application for the first time, the
+second run will some times have a different load address, but subsequent
+runs of the same EFI application will have the same load address as the
+second run. And its a near certainty that if the GRUB EFI binary has changed,
+eg. been recompiled, the load address will also be different.
+
+This ability to predict what the load address will be allows one to assume
+the load address on subsequent runs and thus load the symbols before GRUB2
+starts. The following command illustrates this, assuming that QEMU is
+running and waiting for a debugger connection and the current working
+directory is where @file{gdb_grub} resides:
+
+@example
+gdb -x gdb_grub -ex 'dynamic_load_symbols @var{address of .text section}'
+@end example
+
+If you load the symbols in this manner and, after continuing execution, do
+not see output showing the loading of modules symbol, then its very likely
+that the load address was incorrect.
+
+Another thing to be aware of is how the loading of the GRUB image by the
+firmware affects previously set software breakpoints. On x86 platforms,
+software breakpoints are implemented by GDB by writing a special processor
+instruction at the location of the desired breakpoint. This special instruction
+when executed will stop the program execution and hand control to the
+debugger, GDB. GDB will first saves the instruction bytes that will be
+overwritten at the breakpoint, and will put them back when the breakpoint
+is hit. If GRUB is being run for the first time in QEMU, the firmware will
+be loading the GRUB image into memory where every byte is already set to 0.
+This means that if a breakpoint is set before GRUB is loaded, GDB will save
+the 0-byte(s) where the the special instruction will go. Then when the firmware
+loads the GRUB image and because it is unaware of the debugger, it will
+write the GRUB image to memory, overwriting anything that was there previously,
+notably in this case the instruction that implements the software breakpoint.
+This will be confusing for the person using GDB because GDB will show the
+breakpoint as set, but the brekapoint will never be hit. Furthermore, GDB
+then become confused, such that even deleting an recreating the breakpoint
+will not create usable breakpoints. The @file{gdb_grub} script takes care of
+this by saving the breakpoints just before they are overwritten, and then
+restores them at the start of GRUB execution. So breakpoints for GRUB can be
+set before GRUB is loaded, but be mindful of this effect if you are confused
+as to why breakpoints are not getting hit.
+
+Also note, that hardware breakpoints do not suffer this problem. They are
+implemented by having the breakpoint address in special debug registers on
+the CPU. So they can always be set freely without regard to whether GRUB has
+been loaded or not. The reason that hardware breakpoints aren't always used
+is because there are a limited number of them, usually around 4 on various
+CPUs, and specifically exactly 4 for x86 CPUs. The @file{gdb_grub} script
+goes out of its way to not use hardware breakpoints internally and when
+needed use them as short a time as possible, thus allowing the user to have a
+maximal number at their disposal.
+
+
+@node OVMF debug log
+@subsection OVMF debug log
+
+In order to get the GRUB2 load address from OVMF, first, a debug build
+of OVMF must be obtained (@uref{https://github.com/retrage/edk2-nightly/raw/master/bin/DEBUGX64_OVMF.fd,
+here is one} which is not officially recommended). OVMF will output debug
+messages to a special serial device, which we must add to QEMU. The following
+QEMU command will run the debug OVMF and write the debug messages to a
+file named @file{debug.log}. It is assumed that @file{disk.img} is a disk
+image or block device that is setup to boot GRUB2 EFI.
+
+@example
+qemu-system-x86_64 -bios /path/to/debug/OVMF.fd \
+    -drive file=disk.img,format=raw \
+    -device virtio-scsi-pci,id=scsi0 \
+    -debugcon file:debug.log -global isa-debugcon.iobase=0x402
+@end example
+
+If GRUB2 was started by the (U)EFI firmware, then in the @file{debug.log}
+file one of the last lines should be a log message like:
+@samp{Loading driver at 0x00006AEE000 EntryPoint=0x00006AEE756}. This
+means that the GRUB2 EFI application was loaded at @samp{0x00006AEE000} and
+its .text section is at @samp{0x00006AEE756}.
+
+@node Build GRUB2 to print out the load address
+@subsection Build GRUB2 to print out the load address
+
+GRUB2 can be specially built to output the address of its .text section in
+memory by using the @samp{--enable-efi-debug} configure option. The benefit
+of this method is that it will work on non-virtualized hardware where the
+(U)EFI firmware may not be modifiable. This option has no effect when booting
+with Secure Boot enabled. Otherwise, GRUB will print the gdb command to use
+very early in GRUB startup. The text quickly gets overwritten, perhaps even
+too quickly to see when booting with a physical monitor as the only output
+source. For this reason, a command named "gdbinfo" is enabled which will
+print the same output. So the user can get this info at anytime. If GRUB is
+crashing before the commandline can be reached, this will be of no help
+unfortunately.
+
 @node Porting
 @chapter Porting
 
-- 
2.34.1