EVL Kernel Debugging

[Russell Johnson <russell.johnson@kratosdefense.com>]

[-- Attachment #1: Type: text/plain, Size: 2474 bytes --]

Has there been any successful use of kdb or kgdb with the evl kernel?

We are currently using 5.15.98evl-g1541335eef8b, and have not had much luck
in getting kdg or kgdb to work. We see the start of a kdb session, but the
serial port eventually hangs.

We are connecting the unit under test (running evl kernel) over a serial
port to a secondary machine. 

I think we have all the necessary settings n the kernel config for GDB/KDB:

[root@localhost boot]# cat config-5.15.98evl-g1541335eef8b-dirty|grep GDB
CONFIG_CFG80211_REQUIRE_SIGNED_REGDB=y
CONFIG_CFG80211_USE_KERNEL_REGDB_KEYS=y
# CONFIG_SERIAL_KGDB_NMI is not set
# CONFIG_GDB_SCRIPTS is not set
CONFIG_HAVE_ARCH_KGDB=y
CONFIG_KGDB=y
CONFIG_KGDB_HONOUR_BLOCKLIST=y
CONFIG_KGDB_SERIAL_CONSOLE=y
CONFIG_KGDB_TESTS=y
# CONFIG_KGDB_TESTS_ON_BOOT is not set
CONFIG_KGDB_LOW_LEVEL_TRAP=y
CONFIG_KGDB_KDB=y

Our command line is as follows:
BOOT_IMAGE=/vmlinuz-5.15.98evl-g1541335eef8b-dirty
root=UUID=8748ad87-3ef2-48fe-8d3d-fb2ef72a8f13 ro crashkernel=auto fips=1
kgdboc=ttyS0,115200

On the secondary machine, we connect with minicom or screen over the serial
port. 

The first issue is that  magic sysrq over serial (ctrl-a f g with minicom,
for example) doesn't work even with the proper mask written to
/proc/sys/kernel/sysrq (we tried "1", which should enable all magic-sysrq
features). Doing   echo g > /proc/sysrq-trigger from the evl system does
seem to work, but that isn't ideal. We'd rather break in from the secondary
system when the system is hung. We think we have the correct kernel config
for Magic-Sysrq over serial:

[root@localhost boot]# cat config-5.15.98evl-g1541335eef8b-dirty|grep
MAGIC_SYS
CONFIG_MAGIC_SYSRQ=y
CONFIG_MAGIC_SYSRQ_DEFAULT_ENABLE=0x1
CONFIG_MAGIC_SYSRQ_SERIAL=y
CONFIG_MAGIC_SYSRQ_SERIAL_SEQUENCE=""

After the magic sysrq g is issued, the connection via serial port seems to
have kdb content, but the connection is not stable, usually hanging but
sometimes giving a kdb prompt once or twice. One time we were able to issue
the "kgdb" command within kdb and attempt to connect via gdb, but after the
target remote /dev/ttyS0 within gdb, the gdb process just hung.

Do you have any suggestions on debugging a hard hang in the evl environment?
We get a CPU STUCK when restarting an evl-enabled app multiple times, and
one way to get more insight into this problem is with a kernel debugger.
With the kernel debugger not working, it seems difficult to get any
kernel-level insight. 


[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 6739 bytes --]