Using kgdb, kdb and the kernel debugger internals

From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752731AbdEPMRg (ORCPT ); Tue, 16 May 2017 08:17:36 -0400 Received: from ec2-52-27-115-49.us-west-2.compute.amazonaws.com ([52.27.115.49]:59834 "EHLO osg.samsung.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1752709AbdEPMRU (ORCPT ); Tue, 16 May 2017 08:17:20 -0400 X-Amavis-Alert: BAD HEADER SECTION, Duplicate header field: "References" From: Mauro Carvalho Chehab To: Linux Doc Mailing List Cc: Mauro Carvalho Chehab , Mauro Carvalho Chehab , linux-kernel@vger.kernel.org, Jonathan Corbet , David Woodhouse , Brian Norris , Boris Brezillon , Marek Vasut , Richard Weinberger , Cyrille Pitchen , linux-mtd@lists.infradead.org, Jason Wessel , Jani Nikula , "Herton R. Krzesinski" , Markus Heiser , Laura Abbott , Kees Cook , Jessica Yu , Andrey Ryabinin , Nicolas Palix , kgdb-bugreport@lists.sourceforge.net Subject: [PATCH v2 08/53] docs-rst: convert kgdb DocBook to ReST Date: Tue, 16 May 2017 09:16:00 -0300 Message-Id: <7fb2e8a49037099c56ea209aaa8527e5f3e742ba.1494935649.git.mchehab@s-opensource.com> X-Mailer: git-send-email 2.9.3 In-Reply-To: References: In-Reply-To: References: Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Use pandoc to convert documentation to ReST by calling Documentation/sphinx/tmplcvt script. Signed-off-by: Mauro Carvalho Chehab --- Documentation/DocBook/Makefile | 2 +- Documentation/DocBook/kgdb.tmpl | 918 ------------------------------------- Documentation/dev-tools/index.rst | 1 + Documentation/dev-tools/kgdb.rst | 930 ++++++++++++++++++++++++++++++++++++++ 4 files changed, 932 insertions(+), 919 deletions(-) delete mode 100644 Documentation/DocBook/kgdb.tmpl create mode 100644 Documentation/dev-tools/kgdb.rst diff --git a/Documentation/DocBook/Makefile b/Documentation/DocBook/Makefile index 9df94f7c2003..b9d2b88b9905 100644 --- a/Documentation/DocBook/Makefile +++ b/Documentation/DocBook/Makefile @@ -8,7 +8,7 @@ DOCBOOKS := z8530book.xml \ networking.xml \ - filesystems.xml lsm.xml kgdb.xml \ + filesystems.xml lsm.xml \ libata.xml mtdnand.xml librs.xml rapidio.xml \ s390-drivers.xml scsi.xml \ sh.xml w1.xml diff --git a/Documentation/DocBook/kgdb.tmpl b/Documentation/DocBook/kgdb.tmpl deleted file mode 100644 index 856ac20bf367..000000000000 --- a/Documentation/DocBook/kgdb.tmpl +++ /dev/null @@ -1,918 +0,0 @@ - - - - - - Using kgdb, kdb and the kernel debugger internals - - - - Jason - Wessel - -

- jason.wessel@windriver.com -

- - - - - 2008,2010 - Wind River Systems, Inc. - - - 2004-2005 - MontaVista Software, Inc. - - - 2004 - Amit S. Kale - - - - - This file is licensed under the terms of the GNU General Public License - version 2. This program is licensed "as is" without any warranty of any - kind, whether express or implied. - - - - - - - - Introduction - - The kernel has two different debugger front ends (kdb and kgdb) - which interface to the debug core. It is possible to use either - of the debugger front ends and dynamically transition between them - if you configure the kernel properly at compile and runtime. - - - Kdb is simplistic shell-style interface which you can use on a - system console with a keyboard or serial console. You can use it - to inspect memory, registers, process lists, dmesg, and even set - breakpoints to stop in a certain location. Kdb is not a source - level debugger, although you can set breakpoints and execute some - basic kernel run control. Kdb is mainly aimed at doing some - analysis to aid in development or diagnosing kernel problems. You - can access some symbols by name in kernel built-ins or in kernel - modules if the code was built - with CONFIG_KALLSYMS. - - - Kgdb is intended to be used as a source level debugger for the - Linux kernel. It is used along with gdb to debug a Linux kernel. - The expectation is that gdb can be used to "break in" to the - kernel to inspect memory, variables and look through call stack - information similar to the way an application developer would use - gdb to debug an application. It is possible to place breakpoints - in kernel code and perform some limited execution stepping. - - - Two machines are required for using kgdb. One of these machines is - a development machine and the other is the target machine. The - kernel to be debugged runs on the target machine. The development - machine runs an instance of gdb against the vmlinux file which - contains the symbols (not a boot image such as bzImage, zImage, - uImage...). In gdb the developer specifies the connection - parameters and connects to kgdb. The type of connection a - developer makes with gdb depends on the availability of kgdb I/O - modules compiled as built-ins or loadable kernel modules in the test - machine's kernel. - - - - Compiling a kernel - - - In order to enable compilation of kdb, you must first enable kgdb. - The kgdb test compile options are described in the kgdb test suite chapter. - - - - Kernel config options for kgdb - - To enable CONFIG_KGDB you should look under - "Kernel hacking" / "Kernel debugging" and select "KGDB: kernel debugger". - - - While it is not a hard requirement that you have symbols in your - vmlinux file, gdb tends not to be very useful without the symbolic - data, so you will want to turn - on CONFIG_DEBUG_INFO which is called "Compile the - kernel with debug info" in the config menu. - - - It is advised, but not required, that you turn on the - CONFIG_FRAME_POINTER kernel option which is called "Compile the - kernel with frame pointers" in the config menu. This option - inserts code to into the compiled executable which saves the frame - information in registers or on the stack at different points which - allows a debugger such as gdb to more accurately construct - stack back traces while debugging the kernel. - - - If the architecture that you are using supports the kernel option - CONFIG_STRICT_KERNEL_RWX, you should consider turning it off. This - option will prevent the use of software breakpoints because it - marks certain regions of the kernel's memory space as read-only. - If kgdb supports it for the architecture you are using, you can - use hardware breakpoints if you desire to run with the - CONFIG_STRICT_KERNEL_RWX option turned on, else you need to turn off - this option. - - - Next you should choose one of more I/O drivers to interconnect - debugging host and debugged target. Early boot debugging requires - a KGDB I/O driver that supports early debugging and the driver - must be built into the kernel directly. Kgdb I/O driver - configuration takes place via kernel or module parameters which - you can learn more about in the in the section that describes the - parameter "kgdboc". - - Here is an example set of .config symbols to enable or - disable for kgdb: - - # CONFIG_STRICT_KERNEL_RWX is not set - CONFIG_FRAME_POINTER=y - CONFIG_KGDB=y - CONFIG_KGDB_SERIAL_CONSOLE=y - - - - - Kernel config options for kdb - Kdb is quite a bit more complex than the simple gdbstub - sitting on top of the kernel's debug core. Kdb must implement a - shell, and also adds some helper functions in other parts of the - kernel, responsible for printing out interesting data such as what - you would see if you ran "lsmod", or "ps". In order to build kdb - into the kernel you follow the same steps as you would for kgdb. - - The main config option for kdb - is CONFIG_KGDB_KDB which is called "KGDB_KDB: - include kdb frontend for kgdb" in the config menu. In theory you - would have already also selected an I/O driver such as the - CONFIG_KGDB_SERIAL_CONSOLE interface if you plan on using kdb on a - serial port, when you were configuring kgdb. - - If you want to use a PS/2-style keyboard with kdb, you would - select CONFIG_KDB_KEYBOARD which is called "KGDB_KDB: keyboard as - input device" in the config menu. The CONFIG_KDB_KEYBOARD option - is not used for anything in the gdb interface to kgdb. The - CONFIG_KDB_KEYBOARD option only works with kdb. - - Here is an example set of .config symbols to enable/disable kdb: - - # CONFIG_STRICT_KERNEL_RWX is not set - CONFIG_FRAME_POINTER=y - CONFIG_KGDB=y - CONFIG_KGDB_SERIAL_CONSOLE=y - CONFIG_KGDB_KDB=y - CONFIG_KDB_KEYBOARD=y - - - - - - Kernel Debugger Boot Arguments - This section describes the various runtime kernel - parameters that affect the configuration of the kernel debugger. - The following chapter covers using kdb and kgdb as well as - providing some examples of the configuration parameters. - - Kernel parameter: kgdboc - The kgdboc driver was originally an abbreviation meant to - stand for "kgdb over console". Today it is the primary mechanism - to configure how to communicate from gdb to kgdb as well as the - devices you want to use to interact with the kdb shell. - - For kgdb/gdb, kgdboc is designed to work with a single serial - port. It is intended to cover the circumstance where you want to - use a serial console as your primary console as well as using it to - perform kernel debugging. It is also possible to use kgdb on a - serial port which is not designated as a system console. Kgdboc - may be configured as a kernel built-in or a kernel loadable module. - You can only make use of kgdbwait and early - debugging if you build kgdboc into the kernel as a built-in. - - Optionally you can elect to activate kms (Kernel Mode - Setting) integration. When you use kms with kgdboc and you have a - video driver that has atomic mode setting hooks, it is possible to - enter the debugger on the graphics console. When the kernel - execution is resumed, the previous graphics mode will be restored. - This integration can serve as a useful tool to aid in diagnosing - crashes or doing analysis of memory with kdb while allowing the - full graphics console applications to run. - - - kgdboc arguments - Usage: kgdboc=[kms][[,]kbd][[,]serial_device][,baud] - The order listed above must be observed if you use any of the - optional configurations together. - - Abbreviations: - - kms = Kernel Mode Setting - kbd = Keyboard - - - You can configure kgdboc to use the keyboard, and/or a serial - device depending on if you are using kdb and/or kgdb, in one of the - following scenarios. The order listed above must be observed if - you use any of the optional configurations together. Using kms + - only gdb is generally not a useful combination. - - Using loadable module or built-in - - - As a kernel built-in: - Use the kernel boot argument: kgdboc=<tty-device>,[baud] - - As a kernel loadable module: - Use the command: modprobe kgdboc kgdboc=<tty-device>,[baud] - Here are two examples of how you might format the kgdboc - string. The first is for an x86 target using the first serial port. - The second example is for the ARM Versatile AB using the second - serial port. - - kgdboc=ttyS0,115200 - kgdboc=ttyAMA1,115200 - - - - - - - Configure kgdboc at runtime with sysfs - At run time you can enable or disable kgdboc by echoing a - parameters into the sysfs. Here are two examples: - - Enable kgdboc on ttyS0 - echo ttyS0 > /sys/module/kgdboc/parameters/kgdboc - Disable kgdboc - echo "" > /sys/module/kgdboc/parameters/kgdboc - - NOTE: You do not need to specify the baud if you are - configuring the console on tty which is already configured or - open. - - - More examples - You can configure kgdboc to use the keyboard, and/or a serial device - depending on if you are using kdb and/or kgdb, in one of the - following scenarios. - - kdb and kgdb over only a serial port - kgdboc=<serial_device>[,baud] - Example: kgdboc=ttyS0,115200 - - kdb and kgdb with keyboard and a serial port - kgdboc=kbd,<serial_device>[,baud] - Example: kgdboc=kbd,ttyS0,115200 - - kdb with a keyboard - kgdboc=kbd - - kdb with kernel mode setting - kgdboc=kms,kbd - - kdb with kernel mode setting and kgdb over a serial port - kgdboc=kms,kbd,ttyS0,115200 - - - - NOTE: Kgdboc does not support interrupting the target via the - gdb remote protocol. You must manually send a sysrq-g unless you - have a proxy that splits console output to a terminal program. - A console proxy has a separate TCP port for the debugger and a separate - TCP port for the "human" console. The proxy can take care of sending - the sysrq-g for you. - - When using kgdboc with no debugger proxy, you can end up - connecting the debugger at one of two entry points. If an - exception occurs after you have loaded kgdboc, a message should - print on the console stating it is waiting for the debugger. In - this case you disconnect your terminal program and then connect the - debugger in its place. If you want to interrupt the target system - and forcibly enter a debug session you have to issue a Sysrq - sequence and then type the letter g. Then - you disconnect the terminal session and connect gdb. Your options - if you don't like this are to hack gdb to send the sysrq-g for you - as well as on the initial connect, or to use a debugger proxy that - allows an unmodified gdb to do the debugging. - - - - - - Kernel parameter: kgdbwait - - The Kernel command line option kgdbwait makes - kgdb wait for a debugger connection during booting of a kernel. You - can only use this option if you compiled a kgdb I/O driver into the - kernel and you specified the I/O driver configuration as a kernel - command line option. The kgdbwait parameter should always follow the - configuration parameter for the kgdb I/O driver in the kernel - command line else the I/O driver will not be configured prior to - asking the kernel to use it to wait. - - - The kernel will stop and wait as early as the I/O driver and - architecture allows when you use this option. If you build the - kgdb I/O driver as a loadable kernel module kgdbwait will not do - anything. - - - - Kernel parameter: kgdbcon - The kgdbcon feature allows you to see printk() messages - inside gdb while gdb is connected to the kernel. Kdb does not make - use of the kgdbcon feature. - - Kgdb supports using the gdb serial protocol to send console - messages to the debugger when the debugger is connected and running. - There are two ways to activate this feature. - - Activate with the kernel command line option: - kgdbcon - - Use sysfs before configuring an I/O driver - - echo 1 > /sys/module/kgdb/parameters/kgdb_use_con - - - NOTE: If you do this after you configure the kgdb I/O driver, the - setting will not take effect until the next point the I/O is - reconfigured. - - - - - IMPORTANT NOTE: You cannot use kgdboc + kgdbcon on a tty that is an - active system console. An example of incorrect usage is console=ttyS0,115200 kgdboc=ttyS0 kgdbcon - - It is possible to use this option with kgdboc on a tty that is not a system console. - - - - Run time parameter: kgdbreboot - The kgdbreboot feature allows you to change how the debugger - deals with the reboot notification. You have 3 choices for the - behavior. The default behavior is always set to 0. - - echo -1 > /sys/module/debug_core/parameters/kgdbreboot - Ignore the reboot notification entirely. - - echo 0 > /sys/module/debug_core/parameters/kgdbreboot - Send the detach message to any attached debugger client. - - echo 1 > /sys/module/debug_core/parameters/kgdbreboot - Enter the debugger on reboot notify. - - - - - - Using kdb - - - - Quick start for kdb on a serial port - This is a quick example of how to use kdb. - - Configure kgdboc at boot using kernel parameters: - - console=ttyS0,115200 kgdboc=ttyS0,115200 - - OR - Configure kgdboc after the kernel has booted; assuming you are using a serial port console: - - echo ttyS0 > /sys/module/kgdboc/parameters/kgdboc - - - - Enter the kernel debugger manually or by waiting for an oops or fault. There are several ways you can enter the kernel debugger manually; all involve using the sysrq-g, which means you must have enabled CONFIG_MAGIC_SYSRQ=y in your kernel config. - - When logged in as root or with a super user session you can run: - echo g > /proc/sysrq-trigger - Example using minicom 2.2 - Press: Control-a - Press: f - Press: g - - When you have telneted to a terminal server that supports sending a remote break - Press: Control-] - Type in:send break - Press: Enter - Press: g - - - - From the kdb prompt you can run the "help" command to see a complete list of the commands that are available. - Some useful commands in kdb include: - - lsmod -- Shows where kernel modules are loaded - ps -- Displays only the active processes - ps A -- Shows all the processes - summary -- Shows kernel version info and memory usage - bt -- Get a backtrace of the current process using dump_stack() - dmesg -- View the kernel syslog buffer - go -- Continue the system - - - - - When you are done using kdb you need to consider rebooting the - system or using the "go" command to resuming normal kernel - execution. If you have paused the kernel for a lengthy period of - time, applications that rely on timely networking or anything to do - with real wall clock time could be adversely affected, so you - should take this into consideration when using the kernel - debugger. - - - - - Quick start for kdb using a keyboard connected console - This is a quick example of how to use kdb with a keyboard. - - Configure kgdboc at boot using kernel parameters: - - kgdboc=kbd - - OR - Configure kgdboc after the kernel has booted: - - echo kbd > /sys/module/kgdboc/parameters/kgdboc - - - - Enter the kernel debugger manually or by waiting for an oops or fault. There are several ways you can enter the kernel debugger manually; all involve using the sysrq-g, which means you must have enabled CONFIG_MAGIC_SYSRQ=y in your kernel config. - - When logged in as root or with a super user session you can run: - echo g > /proc/sysrq-trigger - Example using a laptop keyboard - Press and hold down: Alt - Press and hold down: Fn - Press and release the key with the label: SysRq - Release: Fn - Press and release: g - Release: Alt - - Example using a PS/2 101-key keyboard - Press and hold down: Alt - Press and release the key with the label: SysRq - Press and release: g - Release: Alt - - - - - Now type in a kdb command such as "help", "dmesg", "bt" or "go" to continue kernel execution. - - - - - - Using kgdb / gdb - In order to use kgdb you must activate it by passing - configuration information to one of the kgdb I/O drivers. If you - do not pass any configuration information kgdb will not do anything - at all. Kgdb will only actively hook up to the kernel trap hooks - if a kgdb I/O driver is loaded and configured. If you unconfigure - a kgdb I/O driver, kgdb will unregister all the kernel hook points. - - All kgdb I/O drivers can be reconfigured at run time, if - CONFIG_SYSFS and CONFIG_MODULES - are enabled, by echo'ing a new config string to - /sys/module/<driver>/parameter/<option>. - The driver can be unconfigured by passing an empty string. You cannot - change the configuration while the debugger is attached. Make sure - to detach the debugger with the detach command - prior to trying to unconfigure a kgdb I/O driver. - - - Connecting with gdb to a serial port - - Configure kgdboc - Configure kgdboc at boot using kernel parameters: - - kgdboc=ttyS0,115200 - - OR - Configure kgdboc after the kernel has booted: - - echo ttyS0 > /sys/module/kgdboc/parameters/kgdboc - - - - Stop kernel execution (break into the debugger) - In order to connect to gdb via kgdboc, the kernel must - first be stopped. There are several ways to stop the kernel which - include using kgdbwait as a boot argument, via a sysrq-g, or running - the kernel until it takes an exception where it waits for the - debugger to attach. - - When logged in as root or with a super user session you can run: - echo g > /proc/sysrq-trigger - Example using minicom 2.2 - Press: Control-a - Press: f - Press: g - - When you have telneted to a terminal server that supports sending a remote break - Press: Control-] - Type in:send break - Press: Enter - Press: g - - - - - - Connect from gdb - - Example (using a directly connected port): - - - % gdb ./vmlinux - (gdb) set remotebaud 115200 - (gdb) target remote /dev/ttyS0 - - - Example (kgdb to a terminal server on TCP port 2012): - - - % gdb ./vmlinux - (gdb) target remote 192.168.2.2:2012 - - - Once connected, you can debug a kernel the way you would debug an - application program. - - - If you are having problems connecting or something is going - seriously wrong while debugging, it will most often be the case - that you want to enable gdb to be verbose about its target - communications. You do this prior to issuing the target - remote command by typing in: set debug remote 1 - - - - Remember if you continue in gdb, and need to "break in" again, - you need to issue an other sysrq-g. It is easy to create a simple - entry point by putting a breakpoint at sys_sync - and then you can run "sync" from a shell or script to break into the - debugger. - - - - kgdb and kdb interoperability - It is possible to transition between kdb and kgdb dynamically. - The debug core will remember which you used the last time and - automatically start in the same mode. - - Switching between kdb and kgdb - - Switching from kgdb to kdb - - There are two ways to switch from kgdb to kdb: you can use gdb to - issue a maintenance packet, or you can blindly type the command $3#33. - Whenever the kernel debugger stops in kgdb mode it will print the - message KGDB or $3#33 for KDB. It is important - to note that you have to type the sequence correctly in one pass. - You cannot type a backspace or delete because kgdb will interpret - that as part of the debug stream. - - Change from kgdb to kdb by blindly typing: - $3#33 - Change from kgdb to kdb with gdb - maintenance packet 3 - NOTE: Now you must kill gdb. Typically you press control-z and - issue the command: kill -9 % - - - - - Change from kdb to kgdb - There are two ways you can change from kdb to kgdb. You can - manually enter kgdb mode by issuing the kgdb command from the kdb - shell prompt, or you can connect gdb while the kdb shell prompt is - active. The kdb shell looks for the typical first commands that gdb - would issue with the gdb remote protocol and if it sees one of those - commands it automatically changes into kgdb mode. - - From kdb issue the command: - kgdb - Now disconnect your terminal program and connect gdb in its place - At the kdb prompt, disconnect the terminal program and connect gdb in its place. - - - - - Running kdb commands from gdb - It is possible to run a limited set of kdb commands from gdb, - using the gdb monitor command. You don't want to execute any of the - run control or breakpoint operations, because it can disrupt the - state of the kernel debugger. You should be using gdb for - breakpoints and run control operations if you have gdb connected. - The more useful commands to run are things like lsmod, dmesg, ps or - possibly some of the memory information commands. To see all the kdb - commands you can run monitor help. - Example: - -(gdb) monitor ps -1 idle process (state I) and -27 sleeping system daemon (state M) processes suppressed, -use 'ps A' to see all. -Task Addr Pid Parent [*] cpu State Thread Command - -0xc78291d0 1 0 0 0 S 0xc7829404 init -0xc7954150 942 1 0 0 S 0xc7954384 dropbear -0xc78789c0 944 1 0 0 S 0xc7878bf4 sh -(gdb) - - - - - - kgdb Test Suite - - When kgdb is enabled in the kernel config you can also elect to - enable the config parameter KGDB_TESTS. Turning this on will - enable a special kgdb I/O module which is designed to test the - kgdb internal functions. - - - The kgdb tests are mainly intended for developers to test the kgdb - internals as well as a tool for developing a new kgdb architecture - specific implementation. These tests are not really for end users - of the Linux kernel. The primary source of documentation would be - to look in the drivers/misc/kgdbts.c file. - - - The kgdb test suite can also be configured at compile time to run - the core set of tests by setting the kernel config parameter - KGDB_TESTS_ON_BOOT. This particular option is aimed at automated - regression testing and does not require modifying the kernel boot - config arguments. If this is turned on, the kgdb test suite can - be disabled by specifying "kgdbts=" as a kernel boot argument. - - - - Kernel Debugger Internals - - Architecture Specifics - - The kernel debugger is organized into a number of components: - - The debug core - - The debug core is found in kernel/debugger/debug_core.c. It contains: - - A generic OS exception handler which includes - sync'ing the processors into a stopped state on an multi-CPU - system. - The API to talk to the kgdb I/O drivers - The API to make calls to the arch-specific kgdb implementation - The logic to perform safe memory reads and writes to memory while using the debugger - A full implementation for software breakpoints unless overridden by the arch - The API to invoke either the kdb or kgdb frontend to the debug core. - The structures and callback API for atomic kernel mode setting. - NOTE: kgdboc is where the kms callbacks are invoked. - - - - kgdb arch-specific implementation - - This implementation is generally found in arch/*/kernel/kgdb.c. - As an example, arch/x86/kernel/kgdb.c contains the specifics to - implement HW breakpoint as well as the initialization to - dynamically register and unregister for the trap handlers on - this architecture. The arch-specific portion implements: - - contains an arch-specific trap catcher which - invokes kgdb_handle_exception() to start kgdb about doing its - work - translation to and from gdb specific packet format to pt_regs - Registration and unregistration of architecture specific trap hooks - Any special exception handling and cleanup - NMI exception handling and cleanup - (optional) HW breakpoints - - - - gdbstub frontend (aka kgdb) - The gdbstub is located in kernel/debug/gdbstub.c. It contains: - - All the logic to implement the gdb serial protocol - - - kdb frontend - The kdb debugger shell is broken down into a number of - components. The kdb core is located in kernel/debug/kdb. There - are a number of helper functions in some of the other kernel - components to make it possible for kdb to examine and report - information about the kernel without taking locks that could - cause a kernel deadlock. The kdb core contains implements the following functionality. - - A simple shell - The kdb core command set - A registration API to register additional kdb shell commands. - - A good example of a self-contained kdb module - is the "ftdump" command for dumping the ftrace buffer. See: - kernel/trace/trace_kdb.c - For an example of how to dynamically register - a new kdb command you can build the kdb_hello.ko kernel module - from samples/kdb/kdb_hello.c. To build this example you can - set CONFIG_SAMPLES=y and CONFIG_SAMPLE_KDB=m in your kernel - config. Later run "modprobe kdb_hello" and the next time you - enter the kdb shell, you can run the "hello" - command. - - The implementation for kdb_printf() which - emits messages directly to I/O drivers, bypassing the kernel - log. - SW / HW breakpoint management for the kdb shell - - - kgdb I/O driver - - Each kgdb I/O driver has to provide an implementation for the following: - - configuration via built-in or module - dynamic configuration and kgdb hook registration calls - read and write character interface - A cleanup handler for unconfiguring from the kgdb core - (optional) Early debug methodology - - Any given kgdb I/O driver has to operate very closely with the - hardware and must do it in such a way that does not enable - interrupts or change other parts of the system context without - completely restoring them. The kgdb core will repeatedly "poll" - a kgdb I/O driver for characters when it needs input. The I/O - driver is expected to return immediately if there is no data - available. Doing so allows for the future possibility to touch - watchdog hardware in such a way as to have a target system not - reset when these are enabled. - - - - - - If you are intent on adding kgdb architecture specific support - for a new architecture, the architecture should define - HAVE_ARCH_KGDB in the architecture specific - Kconfig file. This will enable kgdb for the architecture, and - at that point you must create an architecture specific kgdb - implementation. - - - There are a few flags which must be set on every architecture in - their <asm/kgdb.h> file. These are: - - - - NUMREGBYTES: The size in bytes of all of the registers, so - that we can ensure they will all fit into a packet. - - - - - BUFMAX: The size in bytes of the buffer GDB will read into. - This must be larger than NUMREGBYTES. - - - - - CACHE_FLUSH_IS_SAFE: Set to 1 if it is always safe to call - flush_cache_range or flush_icache_range. On some architectures, - these functions may not be safe to call on SMP since we keep other - CPUs in a holding pattern. - - - - - - There are also the following functions for the common backend, - found in kernel/kgdb.c, that must be supplied by the - architecture-specific backend unless marked as (optional), in - which case a default function maybe used if the architecture - does not need to provide a specific implementation. - -!Iinclude/linux/kgdb.h - - - kgdboc internals - - kgdboc and uarts - - The kgdboc driver is actually a very thin driver that relies on the - underlying low level to the hardware driver having "polling hooks" - to which the tty driver is attached. In the initial - implementation of kgdboc the serial_core was changed to expose a - low level UART hook for doing polled mode reading and writing of a - single character while in an atomic context. When kgdb makes an I/O - request to the debugger, kgdboc invokes a callback in the serial - core which in turn uses the callback in the UART driver. - - When using kgdboc with a UART, the UART driver must implement two callbacks in the struct uart_ops. Example from drivers/8250.c: -#ifdef CONFIG_CONSOLE_POLL - .poll_get_char = serial8250_get_poll_char, - .poll_put_char = serial8250_put_poll_char, -#endif - - Any implementation specifics around creating a polling driver use the - #ifdef CONFIG_CONSOLE_POLL, as shown above. - Keep in mind that polling hooks have to be implemented in such a way - that they can be called from an atomic context and have to restore - the state of the UART chip on return such that the system can return - to normal when the debugger detaches. You need to be very careful - with any kind of lock you consider, because failing here is most likely - going to mean pressing the reset button. - - - - kgdboc and keyboards - The kgdboc driver contains logic to configure communications - with an attached keyboard. The keyboard infrastructure is only - compiled into the kernel when CONFIG_KDB_KEYBOARD=y is set in the - kernel configuration. - The core polled keyboard driver driver for PS/2 type keyboards - is in drivers/char/kdb_keyboard.c. This driver is hooked into the - debug core when kgdboc populates the callback in the array - called kdb_poll_funcs[]. The - kdb_get_kbd_char() is the top-level function which polls hardware - for single character input. - - - - kgdboc and kms - The kgdboc driver contains logic to request the graphics - display to switch to a text context when you are using - "kgdboc=kms,kbd", provided that you have a video driver which has a - frame buffer console and atomic kernel mode setting support. - - Every time the kernel - debugger is entered it calls kgdboc_pre_exp_handler() which in turn - calls con_debug_enter() in the virtual console layer. On resuming kernel - execution, the kernel debugger calls kgdboc_post_exp_handler() which - in turn calls con_debug_leave(). - Any video driver that wants to be compatible with the kernel - debugger and the atomic kms callbacks must implement the - mode_set_base_atomic, fb_debug_enter and fb_debug_leave operations. - For the fb_debug_enter and fb_debug_leave the option exists to use - the generic drm fb helper functions or implement something custom for - the hardware. The following example shows the initialization of the - .mode_set_base_atomic operation in - drivers/gpu/drm/i915/intel_display.c: - - -static const struct drm_crtc_helper_funcs intel_helper_funcs = { -[...] - .mode_set_base_atomic = intel_pipe_set_base_atomic, -[...] -}; - - - - Here is an example of how the i915 driver initializes the fb_debug_enter and fb_debug_leave functions to use the generic drm helpers in - drivers/gpu/drm/i915/intel_fb.c: - - -static struct fb_ops intelfb_ops = { -[...] - .fb_debug_enter = drm_fb_helper_debug_enter, - .fb_debug_leave = drm_fb_helper_debug_leave, -[...] -}; - - - - - - - - Credits - - The following people have contributed to this document: - - Amit Kaleamitkale@linsyssoft.com - Tom Rinitrini@kernel.crashing.org - - In March 2008 this document was completely rewritten by: - - Jason Wesseljason.wessel@windriver.com - - In Jan 2010 this document was updated to include kdb. - - Jason Wesseljason.wessel@windriver.com - - - - - diff --git a/Documentation/dev-tools/index.rst b/Documentation/dev-tools/index.rst index 07d881147ef3..4ac991dbddb7 100644 --- a/Documentation/dev-tools/index.rst +++ b/Documentation/dev-tools/index.rst @@ -23,6 +23,7 @@ whole; patches welcome! kmemleak kmemcheck gdb-kernel-debugging + kgdb .. only:: subproject and html diff --git a/Documentation/dev-tools/kgdb.rst b/Documentation/dev-tools/kgdb.rst new file mode 100644 index 000000000000..ea01541806c8 --- /dev/null +++ b/Documentation/dev-tools/kgdb.rst @@ -0,0 +1,930 @@ +================================================= +Using kgdb, kdb and the kernel debugger internals +================================================= + +:Author: Jason Wessel + +Introduction +============ + +The kernel has two different debugger front ends (kdb and kgdb) which +interface to the debug core. It is possible to use either of the +debugger front ends and dynamically transition between them if you +configure the kernel properly at compile and runtime. + +Kdb is simplistic shell-style interface which you can use on a system +console with a keyboard or serial console. You can use it to inspect +memory, registers, process lists, dmesg, and even set breakpoints to +stop in a certain location. Kdb is not a source level debugger, although +you can set breakpoints and execute some basic kernel run control. Kdb +is mainly aimed at doing some analysis to aid in development or +diagnosing kernel problems. You can access some symbols by name in +kernel built-ins or in kernel modules if the code was built with +``CONFIG_KALLSYMS``. + +Kgdb is intended to be used as a source level debugger for the Linux +kernel. It is used along with gdb to debug a Linux kernel. The +expectation is that gdb can be used to "break in" to the kernel to +inspect memory, variables and look through call stack information +similar to the way an application developer would use gdb to debug an +application. It is possible to place breakpoints in kernel code and +perform some limited execution stepping. + +Two machines are required for using kgdb. One of these machines is a +development machine and the other is the target machine. The kernel to +be debugged runs on the target machine. The development machine runs an +instance of gdb against the vmlinux file which contains the symbols (not +a boot image such as bzImage, zImage, uImage...). In gdb the developer +specifies the connection parameters and connects to kgdb. The type of +connection a developer makes with gdb depends on the availability of +kgdb I/O modules compiled as built-ins or loadable kernel modules in the +test machine's kernel. + +Compiling a kernel +================== + +- In order to enable compilation of kdb, you must first enable kgdb. + +- The kgdb test compile options are described in the kgdb test suite + chapter. + +Kernel config options for kgdb +------------------------------ + +To enable ``CONFIG_KGDB`` you should look under "Kernel hacking" / +"Kernel debugging" and select "KGDB: kernel debugger". + +While it is not a hard requirement that you have symbols in your vmlinux +file, gdb tends not to be very useful without the symbolic data, so you +will want to turn on ``CONFIG_DEBUG_INFO`` which is called "Compile the +kernel with debug info" in the config menu. + +It is advised, but not required, that you turn on the +``CONFIG_FRAME_POINTER`` kernel option which is called "Compile the +kernel with frame pointers" in the config menu. This option inserts code +to into the compiled executable which saves the frame information in +registers or on the stack at different points which allows a debugger +such as gdb to more accurately construct stack back traces while +debugging the kernel. + +If the architecture that you are using supports the kernel option +CONFIG_STRICT_KERNEL_RWX, you should consider turning it off. This +option will prevent the use of software breakpoints because it marks +certain regions of the kernel's memory space as read-only. If kgdb +supports it for the architecture you are using, you can use hardware +breakpoints if you desire to run with the CONFIG_STRICT_KERNEL_RWX +option turned on, else you need to turn off this option. + +Next you should choose one of more I/O drivers to interconnect debugging +host and debugged target. Early boot debugging requires a KGDB I/O +driver that supports early debugging and the driver must be built into +the kernel directly. Kgdb I/O driver configuration takes place via +kernel or module parameters which you can learn more about in the in the +section that describes the parameter "kgdboc". + +Here is an example set of .config symbols to enable or disable for kgdb: + +- # CONFIG_STRICT_KERNEL_RWX is not set + +- CONFIG_FRAME_POINTER=y + +- CONFIG_KGDB=y + +- CONFIG_KGDB_SERIAL_CONSOLE=y + +Kernel config options for kdb +----------------------------- + +Kdb is quite a bit more complex than the simple gdbstub sitting on top +of the kernel's debug core. Kdb must implement a shell, and also adds +some helper functions in other parts of the kernel, responsible for +printing out interesting data such as what you would see if you ran +"lsmod", or "ps". In order to build kdb into the kernel you follow the +same steps as you would for kgdb. + +The main config option for kdb is ``CONFIG_KGDB_KDB`` which is called +"KGDB_KDB: include kdb frontend for kgdb" in the config menu. In theory +you would have already also selected an I/O driver such as the +CONFIG_KGDB_SERIAL_CONSOLE interface if you plan on using kdb on a +serial port, when you were configuring kgdb. + +If you want to use a PS/2-style keyboard with kdb, you would select +CONFIG_KDB_KEYBOARD which is called "KGDB_KDB: keyboard as input +device" in the config menu. The CONFIG_KDB_KEYBOARD option is not used +for anything in the gdb interface to kgdb. The CONFIG_KDB_KEYBOARD +option only works with kdb. + +Here is an example set of .config symbols to enable/disable kdb: + +- # CONFIG_STRICT_KERNEL_RWX is not set + +- CONFIG_FRAME_POINTER=y + +- CONFIG_KGDB=y + +- CONFIG_KGDB_SERIAL_CONSOLE=y + +- CONFIG_KGDB_KDB=y + +- CONFIG_KDB_KEYBOARD=y + +Kernel Debugger Boot Arguments +============================== + +This section describes the various runtime kernel parameters that affect +the configuration of the kernel debugger. The following chapter covers +using kdb and kgdb as well as providing some examples of the +configuration parameters. + +Kernel parameter: kgdboc +------------------------ + +The kgdboc driver was originally an abbreviation meant to stand for +"kgdb over console". Today it is the primary mechanism to configure how +to communicate from gdb to kgdb as well as the devices you want to use +to interact with the kdb shell. + +For kgdb/gdb, kgdboc is designed to work with a single serial port. It +is intended to cover the circumstance where you want to use a serial +console as your primary console as well as using it to perform kernel +debugging. It is also possible to use kgdb on a serial port which is not +designated as a system console. Kgdboc may be configured as a kernel +built-in or a kernel loadable module. You can only make use of +``kgdbwait`` and early debugging if you build kgdboc into the kernel as +a built-in. + +Optionally you can elect to activate kms (Kernel Mode Setting) +integration. When you use kms with kgdboc and you have a video driver +that has atomic mode setting hooks, it is possible to enter the debugger +on the graphics console. When the kernel execution is resumed, the +previous graphics mode will be restored. This integration can serve as a +useful tool to aid in diagnosing crashes or doing analysis of memory +with kdb while allowing the full graphics console applications to run. + +kgdboc arguments +~~~~~~~~~~~~~~~~ + +Usage: ``kgdboc=[kms][[,]kbd][[,]serial_device][,baud]`` + +The order listed above must be observed if you use any of the optional +configurations together. + +Abbreviations: + +- kms = Kernel Mode Setting + +- kbd = Keyboard + +You can configure kgdboc to use the keyboard, and/or a serial device +depending on if you are using kdb and/or kgdb, in one of the following +scenarios. The order listed above must be observed if you use any of the +optional configurations together. Using kms + only gdb is generally not +a useful combination. + +Using loadable module or built-in +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +1. As a kernel built-in: + + Use the kernel boot argument: ``kgdboc=,[baud]`` + +2. As a kernel loadable module: + + Use the command: ``modprobe kgdboc kgdboc=,[baud]`` + + Here are two examples of how you might format the kgdboc string. The + first is for an x86 target using the first serial port. The second + example is for the ARM Versatile AB using the second serial port. + + 1. ``kgdboc=ttyS0,115200`` + + 2. ``kgdboc=ttyAMA1,115200`` + +Configure kgdboc at runtime with sysfs +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +At run time you can enable or disable kgdboc by echoing a parameters +into the sysfs. Here are two examples: + +1. Enable kgdboc on ttyS0 + + ``echo ttyS0 > /sys/module/kgdboc/parameters/kgdboc`` + +2. Disable kgdboc + + ``echo "" > /sys/module/kgdboc/parameters/kgdboc`` + +NOTE: You do not need to specify the baud if you are configuring the +console on tty which is already configured or open. + +More examples +^^^^^^^^^^^^^ + +You can configure kgdboc to use the keyboard, and/or a serial device +depending on if you are using kdb and/or kgdb, in one of the following +scenarios. + +1. kdb and kgdb over only a serial port + + ``kgdboc=[,baud]`` + + Example: ``kgdboc=ttyS0,115200`` + +2. kdb and kgdb with keyboard and a serial port + + ``kgdboc=kbd,[,baud]`` + + Example: ``kgdboc=kbd,ttyS0,115200`` + +3. kdb with a keyboard + + ``kgdboc=kbd`` + +4. kdb with kernel mode setting + + ``kgdboc=kms,kbd`` + +5. kdb with kernel mode setting and kgdb over a serial port + + ``kgdboc=kms,kbd,ttyS0,115200`` + +NOTE: Kgdboc does not support interrupting the target via the gdb remote +protocol. You must manually send a sysrq-g unless you have a proxy that +splits console output to a terminal program. A console proxy has a +separate TCP port for the debugger and a separate TCP port for the +"human" console. The proxy can take care of sending the sysrq-g for you. + +When using kgdboc with no debugger proxy, you can end up connecting the +debugger at one of two entry points. If an exception occurs after you +have loaded kgdboc, a message should print on the console stating it is +waiting for the debugger. In this case you disconnect your terminal +program and then connect the debugger in its place. If you want to +interrupt the target system and forcibly enter a debug session you have +to issue a Sysrq sequence and then type the letter ``g``. Then you +disconnect the terminal session and connect gdb. Your options if you +don't like this are to hack gdb to send the sysrq-g for you as well as +on the initial connect, or to use a debugger proxy that allows an +unmodified gdb to do the debugging. + +Kernel parameter: kgdbwait +-------------------------- + +The Kernel command line option ``kgdbwait`` makes kgdb wait for a +debugger connection during booting of a kernel. You can only use this +option if you compiled a kgdb I/O driver into the kernel and you +specified the I/O driver configuration as a kernel command line option. +The kgdbwait parameter should always follow the configuration parameter +for the kgdb I/O driver in the kernel command line else the I/O driver +will not be configured prior to asking the kernel to use it to wait. + +The kernel will stop and wait as early as the I/O driver and +architecture allows when you use this option. If you build the kgdb I/O +driver as a loadable kernel module kgdbwait will not do anything. + +Kernel parameter: kgdbcon +------------------------- + +The kgdbcon feature allows you to see printk() messages inside gdb while +gdb is connected to the kernel. Kdb does not make use of the kgdbcon +feature. + +Kgdb supports using the gdb serial protocol to send console messages to +the debugger when the debugger is connected and running. There are two +ways to activate this feature. + +1. Activate with the kernel command line option: + + ``kgdbcon`` + +2. Use sysfs before configuring an I/O driver + + ``echo 1 > /sys/module/kgdb/parameters/kgdb_use_con`` + + NOTE: If you do this after you configure the kgdb I/O driver, the + setting will not take effect until the next point the I/O is + reconfigured. + +IMPORTANT NOTE: You cannot use kgdboc + kgdbcon on a tty that is an +active system console. An example of incorrect usage is +``console=ttyS0,115200 kgdboc=ttyS0 kgdbcon`` + +It is possible to use this option with kgdboc on a tty that is not a +system console. + +Run time parameter: kgdbreboot +------------------------------ + +The kgdbreboot feature allows you to change how the debugger deals with +the reboot notification. You have 3 choices for the behavior. The +default behavior is always set to 0. + +1. echo -1 > /sys/module/debug_core/parameters/kgdbreboot + + Ignore the reboot notification entirely. + +2. echo 0 > /sys/module/debug_core/parameters/kgdbreboot + + Send the detach message to any attached debugger client. + +3. echo 1 > /sys/module/debug_core/parameters/kgdbreboot + + Enter the debugger on reboot notify. + +Using kdb +========= + +Quick start for kdb on a serial port +------------------------------------ + +This is a quick example of how to use kdb. + +1. Configure kgdboc at boot using kernel parameters: + + - ``console=ttyS0,115200 kgdboc=ttyS0,115200`` + + OR + + Configure kgdboc after the kernel has booted; assuming you are using + a serial port console: + + - ``echo ttyS0 > /sys/module/kgdboc/parameters/kgdboc`` + +2. Enter the kernel debugger manually or by waiting for an oops or + fault. There are several ways you can enter the kernel debugger + manually; all involve using the sysrq-g, which means you must have + enabled CONFIG_MAGIC_SYSRQ=y in your kernel config. + + - When logged in as root or with a super user session you can run: + + ``echo g > /proc/sysrq-trigger`` + + - Example using minicom 2.2 + + Press: ``Control-a`` + + Press: ``f`` + + Press: ``g`` + + - When you have telneted to a terminal server that supports sending + a remote break + + Press: ``Control-]`` + + Type in:\ ``send break`` + + Press: ``Enter`` + + Press: ``g`` + +3. From the kdb prompt you can run the "help" command to see a complete + list of the commands that are available. + + Some useful commands in kdb include: + + - lsmod -- Shows where kernel modules are loaded + + - ps -- Displays only the active processes + + - ps A -- Shows all the processes + + - summary -- Shows kernel version info and memory usage + + - bt -- Get a backtrace of the current process using dump_stack() + + - dmesg -- View the kernel syslog buffer + + - go -- Continue the system + +4. When you are done using kdb you need to consider rebooting the system + or using the "go" command to resuming normal kernel execution. If you + have paused the kernel for a lengthy period of time, applications + that rely on timely networking or anything to do with real wall clock + time could be adversely affected, so you should take this into + consideration when using the kernel debugger. + +Quick start for kdb using a keyboard connected console +------------------------------------------------------ + +This is a quick example of how to use kdb with a keyboard. + +1. Configure kgdboc at boot using kernel parameters: + + - ``kgdboc=kbd`` + + OR + + Configure kgdboc after the kernel has booted: + + - ``echo kbd > /sys/module/kgdboc/parameters/kgdboc`` + +2. Enter the kernel debugger manually or by waiting for an oops or + fault. There are several ways you can enter the kernel debugger + manually; all involve using the sysrq-g, which means you must have + enabled CONFIG_MAGIC_SYSRQ=y in your kernel config. + + - When logged in as root or with a super user session you can run: + + ``echo g > /proc/sysrq-trigger`` + + - Example using a laptop keyboard + + Press and hold down: ``Alt`` + + Press and hold down: ``Fn`` + + Press and release the key with the label: ``SysRq`` + + Release: ``Fn`` + + Press and release: ``g`` + + Release: ``Alt`` + + - Example using a PS/2 101-key keyboard + + Press and hold down: ``Alt`` + + Press and release the key with the label: ``SysRq`` + + Press and release: ``g`` + + Release: ``Alt`` + +3. Now type in a kdb command such as "help", "dmesg", "bt" or "go" to + continue kernel execution. + +Using kgdb / gdb +================ + +In order to use kgdb you must activate it by passing configuration +information to one of the kgdb I/O drivers. If you do not pass any +configuration information kgdb will not do anything at all. Kgdb will +only actively hook up to the kernel trap hooks if a kgdb I/O driver is +loaded and configured. If you unconfigure a kgdb I/O driver, kgdb will +unregister all the kernel hook points. + +All kgdb I/O drivers can be reconfigured at run time, if +``CONFIG_SYSFS`` and ``CONFIG_MODULES`` are enabled, by echo'ing a new +config string to ``/sys/module//parameter/