* Backtrace stopped: previous frame identical to this frame (corrupt stack?) , even with fresh qemu and linux build
@ 2022-04-22 12:14 ` Chan Kim
0 siblings, 0 replies; 12+ messages in thread
From: Chan Kim @ 2022-04-22 12:14 UTC (permalink / raw)
To: 'qemu-devel', kernelnewbies
Hello all,
Really strange thing happening here.. I can't see the full stack trace with
'bt' command in gdb.
So I tried with fresh linux-5.10.122 source and qemu-6.2.0 source and it's
happening too!
(it's happening when I do combinations with linux 5.10.0 and qemu-5.1.0. But
it's not happening with linux-5.4.21)
I would be grateful if somebody could check if this happens to other people
or just me.
1. download linux-5.1.122 tarball from https://www.kernel.org/
2. uncompress it and set env variable ARCH=arm64,
CROSS_COMPILE=aarch64-none-elf- , do "make defconfig" and "make -j`nproc`
Image"
3. download qemu-6.2.0 from https://www.qemu.org/
4. uncompress it and do "mkdir build" "cd build" "../configure
--target-list=aarch64-softmmu --enable-debug"
5. run qemu and wait for debugger to attach.
qemu-6.2.0/build/aarch64-softmmu/qemu-system-aarch64 -machine
virt,gic-version=max,secure=off,virtualization=true -cpu max -kernel
linux-5.10.112/arch/arm64/boot/Image -m 2G -nographic -netdev
user,id=vnet,hostfwd=:127.0.0.1:0-:22,tftp=/srv/tftp -device
virtio-net-pci,netdev=vnet -machine iommu=smmuv3 --append "root=/dev/ram
init=/init nokaslr earlycon ip=dhcp hugepages=16" -s -S
6. run debugger, do "aarch64-none-elf-gdb linux-6.10.112/vmlinux -x
gdb_script"
(gdb_script content :
target remote :1234
layout src
b start_kernel
b __driver_attach
)
Now, in gdb, when you press 'c' twice, it'll stop at the first
__driver_attach. (first one stops at start_kernel).
When you are at __attach_driver, type 'bt'. See if you see the full function
stack trace.
This is what I see.
(gdb) bt
#0 __driver_attach (dev=0xffff000002582810, data=0xffff800011dc2358
<dummy_regulator_driver+40>)
at drivers/base/dd.c:1060
#1 0xffff8000107a3ed0 in bus_for_each_dev (bus=<optimized out>,
start=<optimized out>,
data=0xffff800011dc2358 <dummy_regulator_driver+40>,
fn=0xffff8000107a6f60 <__driver_attach>)
at drivers/base/bus.c:305
#2 0xd6d78000107a5c58 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
I used to see more thatn 20 stacks trace but strangely I see only two.
I can still see many stacks for linux-5.4.21 that I was working with in the
past.
Could anyone check if this happens to anyone?
I think if I add BLK_DEV_RAM and set initramfs.cpio.gz in the linux build,
the kernel will boot ok to the shell prompt.
Only the gdb can't show the stack levels.
My OS : ubuntu-20.04 5.13.0-35-generic
$ aarch64-none-elf-gdb --version
GNU gdb (GNU Toolchain for the A-profile Architecture 10.2-2020.11
(arm-10.16)) 10.1.90.20201028-git
Copyright (C) 2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
<http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Thank you.
Chan Kim
_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
^ permalink raw reply [flat|nested] 12+ messages in thread
* Backtrace stopped: previous frame identical to this frame (corrupt stack?) , even with fresh qemu and linux build
@ 2022-04-22 12:14 ` Chan Kim
0 siblings, 0 replies; 12+ messages in thread
From: Chan Kim @ 2022-04-22 12:14 UTC (permalink / raw)
To: 'qemu-devel', kernelnewbies
Hello all,
Really strange thing happening here.. I can't see the full stack trace with
'bt' command in gdb.
So I tried with fresh linux-5.10.122 source and qemu-6.2.0 source and it's
happening too!
(it's happening when I do combinations with linux 5.10.0 and qemu-5.1.0. But
it's not happening with linux-5.4.21)
I would be grateful if somebody could check if this happens to other people
or just me.
1. download linux-5.1.122 tarball from https://www.kernel.org/
2. uncompress it and set env variable ARCH=arm64,
CROSS_COMPILE=aarch64-none-elf- , do "make defconfig" and "make -j`nproc`
Image"
3. download qemu-6.2.0 from https://www.qemu.org/
4. uncompress it and do "mkdir build" "cd build" "../configure
--target-list=aarch64-softmmu --enable-debug"
5. run qemu and wait for debugger to attach.
qemu-6.2.0/build/aarch64-softmmu/qemu-system-aarch64 -machine
virt,gic-version=max,secure=off,virtualization=true -cpu max -kernel
linux-5.10.112/arch/arm64/boot/Image -m 2G -nographic -netdev
user,id=vnet,hostfwd=:127.0.0.1:0-:22,tftp=/srv/tftp -device
virtio-net-pci,netdev=vnet -machine iommu=smmuv3 --append "root=/dev/ram
init=/init nokaslr earlycon ip=dhcp hugepages=16" -s -S
6. run debugger, do "aarch64-none-elf-gdb linux-6.10.112/vmlinux -x
gdb_script"
(gdb_script content :
target remote :1234
layout src
b start_kernel
b __driver_attach
)
Now, in gdb, when you press 'c' twice, it'll stop at the first
__driver_attach. (first one stops at start_kernel).
When you are at __attach_driver, type 'bt'. See if you see the full function
stack trace.
This is what I see.
(gdb) bt
#0 __driver_attach (dev=0xffff000002582810, data=0xffff800011dc2358
<dummy_regulator_driver+40>)
at drivers/base/dd.c:1060
#1 0xffff8000107a3ed0 in bus_for_each_dev (bus=<optimized out>,
start=<optimized out>,
data=0xffff800011dc2358 <dummy_regulator_driver+40>,
fn=0xffff8000107a6f60 <__driver_attach>)
at drivers/base/bus.c:305
#2 0xd6d78000107a5c58 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
I used to see more thatn 20 stacks trace but strangely I see only two.
I can still see many stacks for linux-5.4.21 that I was working with in the
past.
Could anyone check if this happens to anyone?
I think if I add BLK_DEV_RAM and set initramfs.cpio.gz in the linux build,
the kernel will boot ok to the shell prompt.
Only the gdb can't show the stack levels.
My OS : ubuntu-20.04 5.13.0-35-generic
$ aarch64-none-elf-gdb --version
GNU gdb (GNU Toolchain for the A-profile Architecture 10.2-2020.11
(arm-10.16)) 10.1.90.20201028-git
Copyright (C) 2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
<http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Thank you.
Chan Kim
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Backtrace stopped: previous frame identical to this frame (corrupt stack?) , even with fresh qemu and linux build
2022-04-22 12:14 ` Chan Kim
@ 2022-04-22 14:52 ` Mulyadi Santosa
-1 siblings, 0 replies; 12+ messages in thread
From: Mulyadi Santosa @ 2022-04-22 14:52 UTC (permalink / raw)
To: Chan Kim; +Cc: qemu-devel, kernelnewbies
[-- Attachment #1: Type: text/plain, Size: 3598 bytes --]
On Fri, Apr 22, 2022 at 7:30 PM Chan Kim <ckim@etri.re.kr> wrote:
> Hello all,
>
> Really strange thing happening here.. I can't see the full stack trace with
> 'bt' command in gdb.
> So I tried with fresh linux-5.10.122 source and qemu-6.2.0 source and it's
> happening too!
> (it's happening when I do combinations with linux 5.10.0 and qemu-5.1.0.
> But
> it's not happening with linux-5.4.21)
>
> I would be grateful if somebody could check if this happens to other people
> or just me.
>
> 1. download linux-5.1.122 tarball from https://www.kernel.org/
> 2. uncompress it and set env variable ARCH=arm64,
> CROSS_COMPILE=aarch64-none-elf- , do "make defconfig" and "make -j`nproc`
> Image"
> 3. download qemu-6.2.0 from https://www.qemu.org/
> 4. uncompress it and do "mkdir build" "cd build" "../configure
> --target-list=aarch64-softmmu --enable-debug"
> 5. run qemu and wait for debugger to attach.
> qemu-6.2.0/build/aarch64-softmmu/qemu-system-aarch64 -machine
> virt,gic-version=max,secure=off,virtualization=true -cpu max -kernel
> linux-5.10.112/arch/arm64/boot/Image -m 2G -nographic -netdev
> user,id=vnet,hostfwd=:127.0.0.1:0-:22,tftp=/srv/tftp -device
> virtio-net-pci,netdev=vnet -machine iommu=smmuv3 --append "root=/dev/ram
> init=/init nokaslr earlycon ip=dhcp hugepages=16" -s -S
> 6. run debugger, do "aarch64-none-elf-gdb linux-6.10.112/vmlinux -x\
>
It has been long time since I compiled linux kernel but I guess, either you
need to compile kernel with enabled frame pointer, and/or you need to
enable debug symbol embedded into final kernel image. cmiiw
> gdb_script"
> (gdb_script content :
> target remote :1234
> layout src
> b start_kernel
> b __driver_attach
> )
>
> Now, in gdb, when you press 'c' twice, it'll stop at the first
> __driver_attach. (first one stops at start_kernel).
> When you are at __attach_driver, type 'bt'. See if you see the full
> function
> stack trace.
> This is what I see.
> (gdb) bt
> #0 __driver_attach (dev=0xffff000002582810, data=0xffff800011dc2358
> <dummy_regulator_driver+40>)
> at drivers/base/dd.c:1060
> #1 0xffff8000107a3ed0 in bus_for_each_dev (bus=<optimized out>,
> start=<optimized out>,
> data=0xffff800011dc2358 <dummy_regulator_driver+40>,
> fn=0xffff8000107a6f60 <__driver_attach>)
> at drivers/base/bus.c:305
> #2 0xd6d78000107a5c58 in ?? ()
> Backtrace stopped: previous frame identical to this frame (corrupt stack?)
>
> I used to see more thatn 20 stacks trace but strangely I see only two.
> I can still see many stacks for linux-5.4.21 that I was working with in the
> past.
> Could anyone check if this happens to anyone?
> I think if I add BLK_DEV_RAM and set initramfs.cpio.gz in the linux build,
> the kernel will boot ok to the shell prompt.
> Only the gdb can't show the stack levels.
>
> My OS : ubuntu-20.04 5.13.0-35-generic
>
> $ aarch64-none-elf-gdb --version
> GNU gdb (GNU Toolchain for the A-profile Architecture 10.2-2020.11
> (arm-10.16)) 10.1.90.20201028-git
> Copyright (C) 2020 Free Software Foundation, Inc.
> License GPLv3+: GNU GPL version 3 or later
> <http://gnu.org/licenses/gpl.html>
> This is free software: you are free to change and redistribute it.
> There is NO WARRANTY, to the extent permitted by law.
>
> Thank you.
> Chan Kim
>
>
>
>
>
> _______________________________________________
> Kernelnewbies mailing list
> Kernelnewbies@kernelnewbies.org
> https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
>
--
regards,
Mulyadi Santosa
Freelance Linux trainer and consultant
blog: the-hydra.blogspot.com
training: mulyaditraining.blogspot.com
[-- Attachment #2: Type: text/html, Size: 5101 bytes --]
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Backtrace stopped: previous frame identical to this frame (corrupt stack?) , even with fresh qemu and linux build
@ 2022-04-22 14:52 ` Mulyadi Santosa
0 siblings, 0 replies; 12+ messages in thread
From: Mulyadi Santosa @ 2022-04-22 14:52 UTC (permalink / raw)
To: Chan Kim; +Cc: qemu-devel, kernelnewbies
[-- Attachment #1.1: Type: text/plain, Size: 3598 bytes --]
On Fri, Apr 22, 2022 at 7:30 PM Chan Kim <ckim@etri.re.kr> wrote:
> Hello all,
>
> Really strange thing happening here.. I can't see the full stack trace with
> 'bt' command in gdb.
> So I tried with fresh linux-5.10.122 source and qemu-6.2.0 source and it's
> happening too!
> (it's happening when I do combinations with linux 5.10.0 and qemu-5.1.0.
> But
> it's not happening with linux-5.4.21)
>
> I would be grateful if somebody could check if this happens to other people
> or just me.
>
> 1. download linux-5.1.122 tarball from https://www.kernel.org/
> 2. uncompress it and set env variable ARCH=arm64,
> CROSS_COMPILE=aarch64-none-elf- , do "make defconfig" and "make -j`nproc`
> Image"
> 3. download qemu-6.2.0 from https://www.qemu.org/
> 4. uncompress it and do "mkdir build" "cd build" "../configure
> --target-list=aarch64-softmmu --enable-debug"
> 5. run qemu and wait for debugger to attach.
> qemu-6.2.0/build/aarch64-softmmu/qemu-system-aarch64 -machine
> virt,gic-version=max,secure=off,virtualization=true -cpu max -kernel
> linux-5.10.112/arch/arm64/boot/Image -m 2G -nographic -netdev
> user,id=vnet,hostfwd=:127.0.0.1:0-:22,tftp=/srv/tftp -device
> virtio-net-pci,netdev=vnet -machine iommu=smmuv3 --append "root=/dev/ram
> init=/init nokaslr earlycon ip=dhcp hugepages=16" -s -S
> 6. run debugger, do "aarch64-none-elf-gdb linux-6.10.112/vmlinux -x\
>
It has been long time since I compiled linux kernel but I guess, either you
need to compile kernel with enabled frame pointer, and/or you need to
enable debug symbol embedded into final kernel image. cmiiw
> gdb_script"
> (gdb_script content :
> target remote :1234
> layout src
> b start_kernel
> b __driver_attach
> )
>
> Now, in gdb, when you press 'c' twice, it'll stop at the first
> __driver_attach. (first one stops at start_kernel).
> When you are at __attach_driver, type 'bt'. See if you see the full
> function
> stack trace.
> This is what I see.
> (gdb) bt
> #0 __driver_attach (dev=0xffff000002582810, data=0xffff800011dc2358
> <dummy_regulator_driver+40>)
> at drivers/base/dd.c:1060
> #1 0xffff8000107a3ed0 in bus_for_each_dev (bus=<optimized out>,
> start=<optimized out>,
> data=0xffff800011dc2358 <dummy_regulator_driver+40>,
> fn=0xffff8000107a6f60 <__driver_attach>)
> at drivers/base/bus.c:305
> #2 0xd6d78000107a5c58 in ?? ()
> Backtrace stopped: previous frame identical to this frame (corrupt stack?)
>
> I used to see more thatn 20 stacks trace but strangely I see only two.
> I can still see many stacks for linux-5.4.21 that I was working with in the
> past.
> Could anyone check if this happens to anyone?
> I think if I add BLK_DEV_RAM and set initramfs.cpio.gz in the linux build,
> the kernel will boot ok to the shell prompt.
> Only the gdb can't show the stack levels.
>
> My OS : ubuntu-20.04 5.13.0-35-generic
>
> $ aarch64-none-elf-gdb --version
> GNU gdb (GNU Toolchain for the A-profile Architecture 10.2-2020.11
> (arm-10.16)) 10.1.90.20201028-git
> Copyright (C) 2020 Free Software Foundation, Inc.
> License GPLv3+: GNU GPL version 3 or later
> <http://gnu.org/licenses/gpl.html>
> This is free software: you are free to change and redistribute it.
> There is NO WARRANTY, to the extent permitted by law.
>
> Thank you.
> Chan Kim
>
>
>
>
>
> _______________________________________________
> Kernelnewbies mailing list
> Kernelnewbies@kernelnewbies.org
> https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
>
--
regards,
Mulyadi Santosa
Freelance Linux trainer and consultant
blog: the-hydra.blogspot.com
training: mulyaditraining.blogspot.com
[-- Attachment #1.2: Type: text/html, Size: 5101 bytes --]
[-- Attachment #2: Type: text/plain, Size: 170 bytes --]
_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
^ permalink raw reply [flat|nested] 12+ messages in thread
* RE: Backtrace stopped: previous frame identical to this frame (corrupt stack?) , even with fresh qemu and linux build
2022-04-22 14:52 ` Mulyadi Santosa
@ 2022-04-22 23:36 ` Chan Kim
-1 siblings, 0 replies; 12+ messages in thread
From: Chan Kim @ 2022-04-22 23:36 UTC (permalink / raw)
To: 'Mulyadi Santosa'; +Cc: 'qemu-devel', 'kernelnewbies'
[-- Attachment #1.1: Type: text/plain, Size: 4316 bytes --]
Hi, Mulyadi
Thank you for replying.
I found CONFIG_DEBUG_FRAME_POINTER, CONFIG_DEBUG_INFO are already set by default.
And I tried adding CONFIG_DEBUG_KERNEL, CONFIG_KGDB, CONFIG_GDB_SCRIPTS, CONFIG_STACKTRACE all to no avail.
Regards,
Chan
From: Mulyadi Santosa <mulyadi.santosa@gmail.com>
Sent: Friday, April 22, 2022 11:53 PM
To: Chan Kim <ckim@etri.re.kr>
Cc: qemu-devel <qemu-devel@nongnu.org>; kernelnewbies <kernelnewbies@kernelnewbies.org>
Subject: Re: Backtrace stopped: previous frame identical to this frame (corrupt stack?) , even with fresh qemu and linux build
On Fri, Apr 22, 2022 at 7:30 PM Chan Kim <ckim@etri.re.kr <mailto:ckim@etri.re.kr> > wrote:
Hello all,
Really strange thing happening here.. I can't see the full stack trace with
'bt' command in gdb.
So I tried with fresh linux-5.10.122 source and qemu-6.2.0 source and it's
happening too!
(it's happening when I do combinations with linux 5.10.0 and qemu-5.1.0. But
it's not happening with linux-5.4.21)
I would be grateful if somebody could check if this happens to other people
or just me.
1. download linux-5.1.122 tarball from https://www.kernel.org/
2. uncompress it and set env variable ARCH=arm64,
CROSS_COMPILE=aarch64-none-elf- , do "make defconfig" and "make -j`nproc`
Image"
3. download qemu-6.2.0 from https://www.qemu.org/
4. uncompress it and do "mkdir build" "cd build" "../configure
--target-list=aarch64-softmmu --enable-debug"
5. run qemu and wait for debugger to attach.
qemu-6.2.0/build/aarch64-softmmu/qemu-system-aarch64 -machine
virt,gic-version=max,secure=off,virtualization=true -cpu max -kernel
linux-5.10.112/arch/arm64/boot/Image -m 2G -nographic -netdev
user,id=vnet,hostfwd=:127.0.0.1:0-:22,tftp=/srv/tftp -device
virtio-net-pci,netdev=vnet -machine iommu=smmuv3 --append "root=/dev/ram
init=/init nokaslr earlycon ip=dhcp hugepages=16" -s -S
6. run debugger, do "aarch64-none-elf-gdb linux-6.10.112/vmlinux -x\
It has been long time since I compiled linux kernel but I guess, either you need to compile kernel with enabled frame pointer, and/or you need to enable debug symbol embedded into final kernel image. cmiiw
gdb_script"
(gdb_script content :
target remote :1234
layout src
b start_kernel
b __driver_attach
)
Now, in gdb, when you press 'c' twice, it'll stop at the first
__driver_attach. (first one stops at start_kernel).
When you are at __attach_driver, type 'bt'. See if you see the full function
stack trace.
This is what I see.
(gdb) bt
#0 __driver_attach (dev=0xffff000002582810, data=0xffff800011dc2358
<dummy_regulator_driver+40>)
at drivers/base/dd.c:1060
#1 0xffff8000107a3ed0 in bus_for_each_dev (bus=<optimized out>,
start=<optimized out>,
data=0xffff800011dc2358 <dummy_regulator_driver+40>,
fn=0xffff8000107a6f60 <__driver_attach>)
at drivers/base/bus.c:305
#2 0xd6d78000107a5c58 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
I used to see more thatn 20 stacks trace but strangely I see only two.
I can still see many stacks for linux-5.4.21 that I was working with in the
past.
Could anyone check if this happens to anyone?
I think if I add BLK_DEV_RAM and set initramfs.cpio.gz in the linux build,
the kernel will boot ok to the shell prompt.
Only the gdb can't show the stack levels.
My OS : ubuntu-20.04 5.13.0-35-generic
$ aarch64-none-elf-gdb --version
GNU gdb (GNU Toolchain for the A-profile Architecture 10.2-2020.11
(arm-10.16)) 10.1.90.20201028-git
Copyright (C) 2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
<http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Thank you.
Chan Kim
_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org <mailto:Kernelnewbies@kernelnewbies.org>
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
--
regards,
Mulyadi Santosa
Freelance Linux trainer and consultant
blog: the-hydra.blogspot.com <http://the-hydra.blogspot.com>
training: mulyaditraining.blogspot.com <http://mulyaditraining.blogspot.com>
[-- Attachment #1.2: Type: text/html, Size: 9484 bytes --]
[-- Attachment #2: Type: text/plain, Size: 170 bytes --]
_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
^ permalink raw reply [flat|nested] 12+ messages in thread
* RE: Backtrace stopped: previous frame identical to this frame (corrupt stack?) , even with fresh qemu and linux build
@ 2022-04-22 23:36 ` Chan Kim
0 siblings, 0 replies; 12+ messages in thread
From: Chan Kim @ 2022-04-22 23:36 UTC (permalink / raw)
To: 'Mulyadi Santosa'; +Cc: 'qemu-devel', 'kernelnewbies'
[-- Attachment #1: Type: text/plain, Size: 4316 bytes --]
Hi, Mulyadi
Thank you for replying.
I found CONFIG_DEBUG_FRAME_POINTER, CONFIG_DEBUG_INFO are already set by default.
And I tried adding CONFIG_DEBUG_KERNEL, CONFIG_KGDB, CONFIG_GDB_SCRIPTS, CONFIG_STACKTRACE all to no avail.
Regards,
Chan
From: Mulyadi Santosa <mulyadi.santosa@gmail.com>
Sent: Friday, April 22, 2022 11:53 PM
To: Chan Kim <ckim@etri.re.kr>
Cc: qemu-devel <qemu-devel@nongnu.org>; kernelnewbies <kernelnewbies@kernelnewbies.org>
Subject: Re: Backtrace stopped: previous frame identical to this frame (corrupt stack?) , even with fresh qemu and linux build
On Fri, Apr 22, 2022 at 7:30 PM Chan Kim <ckim@etri.re.kr <mailto:ckim@etri.re.kr> > wrote:
Hello all,
Really strange thing happening here.. I can't see the full stack trace with
'bt' command in gdb.
So I tried with fresh linux-5.10.122 source and qemu-6.2.0 source and it's
happening too!
(it's happening when I do combinations with linux 5.10.0 and qemu-5.1.0. But
it's not happening with linux-5.4.21)
I would be grateful if somebody could check if this happens to other people
or just me.
1. download linux-5.1.122 tarball from https://www.kernel.org/
2. uncompress it and set env variable ARCH=arm64,
CROSS_COMPILE=aarch64-none-elf- , do "make defconfig" and "make -j`nproc`
Image"
3. download qemu-6.2.0 from https://www.qemu.org/
4. uncompress it and do "mkdir build" "cd build" "../configure
--target-list=aarch64-softmmu --enable-debug"
5. run qemu and wait for debugger to attach.
qemu-6.2.0/build/aarch64-softmmu/qemu-system-aarch64 -machine
virt,gic-version=max,secure=off,virtualization=true -cpu max -kernel
linux-5.10.112/arch/arm64/boot/Image -m 2G -nographic -netdev
user,id=vnet,hostfwd=:127.0.0.1:0-:22,tftp=/srv/tftp -device
virtio-net-pci,netdev=vnet -machine iommu=smmuv3 --append "root=/dev/ram
init=/init nokaslr earlycon ip=dhcp hugepages=16" -s -S
6. run debugger, do "aarch64-none-elf-gdb linux-6.10.112/vmlinux -x\
It has been long time since I compiled linux kernel but I guess, either you need to compile kernel with enabled frame pointer, and/or you need to enable debug symbol embedded into final kernel image. cmiiw
gdb_script"
(gdb_script content :
target remote :1234
layout src
b start_kernel
b __driver_attach
)
Now, in gdb, when you press 'c' twice, it'll stop at the first
__driver_attach. (first one stops at start_kernel).
When you are at __attach_driver, type 'bt'. See if you see the full function
stack trace.
This is what I see.
(gdb) bt
#0 __driver_attach (dev=0xffff000002582810, data=0xffff800011dc2358
<dummy_regulator_driver+40>)
at drivers/base/dd.c:1060
#1 0xffff8000107a3ed0 in bus_for_each_dev (bus=<optimized out>,
start=<optimized out>,
data=0xffff800011dc2358 <dummy_regulator_driver+40>,
fn=0xffff8000107a6f60 <__driver_attach>)
at drivers/base/bus.c:305
#2 0xd6d78000107a5c58 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
I used to see more thatn 20 stacks trace but strangely I see only two.
I can still see many stacks for linux-5.4.21 that I was working with in the
past.
Could anyone check if this happens to anyone?
I think if I add BLK_DEV_RAM and set initramfs.cpio.gz in the linux build,
the kernel will boot ok to the shell prompt.
Only the gdb can't show the stack levels.
My OS : ubuntu-20.04 5.13.0-35-generic
$ aarch64-none-elf-gdb --version
GNU gdb (GNU Toolchain for the A-profile Architecture 10.2-2020.11
(arm-10.16)) 10.1.90.20201028-git
Copyright (C) 2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
<http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Thank you.
Chan Kim
_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org <mailto:Kernelnewbies@kernelnewbies.org>
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
--
regards,
Mulyadi Santosa
Freelance Linux trainer and consultant
blog: the-hydra.blogspot.com <http://the-hydra.blogspot.com>
training: mulyaditraining.blogspot.com <http://mulyaditraining.blogspot.com>
[-- Attachment #2: Type: text/html, Size: 9484 bytes --]
^ permalink raw reply [flat|nested] 12+ messages in thread
* RE: Backtrace stopped: previous frame identical to this frame (corrupt stack?) , even with fresh qemu and linux build
2022-04-22 14:52 ` Mulyadi Santosa
@ 2022-04-26 1:46 ` Chan Kim
-1 siblings, 0 replies; 12+ messages in thread
From: Chan Kim @ 2022-04-26 1:46 UTC (permalink / raw)
To: 'qemu-devel', 'kernelnewbies'
[-- Attachment #1.1: Type: text/plain, Size: 6552 bytes --]
Hello,
I hope somebody could check this case. It’s easily reproducible for anybody working with qemu and arm64 linux.
I returned to this problem and made another observation.(showing the back-trace is really broken).
In another incident of breakpoint at function __driver_attach, (right after it stopped at the function)
(gdb) bt
#0 __driver_attach (dev=dev@entry=0xffff0000401d1810, data=data@entry=0xffff800011bbbbb8 <mxc_gpio_driver+40>) at drivers/base/dd.c:1046
#1 0xffff8000107684f8 in bus_for_each_dev (bus=0xffff800011cba910 <platform_bus_type>, start=0x0, data=0xffff800011bbbbb8 <mxc_gpio_driver+40>, fn=0xffff80001076b860 <__driver_attach>) at drivers/base/bus.c:307
#2 0xb8cd80001076a594 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
(gdb) x/5g $sp
0xffff800011dcbcc0: 0xffff800011dcbd20 0xb8cd80001076a594
0xffff800011dcbcd0: 0xffff80001076b860 0xffff800011bbbbb8
0xffff800011dcbce0: 0x0000000000000000
Because it's right after the pc reached the function __driver_attach, the sp is still not updated from previous function (bus_for_each_dev). And the first two values at the $sp are supposed to be the fp and lr of the previous function (see https://stackoverflow.com/questions/66098678/understanding-aarch64-assembly-function-call-how-is-stack-operated arm64 stores previous function's fp and lr at the bottom of new stack frame as it enters a function). The lr (link register, the address to return after this bus_for_each_dev function) is really 0xb8cd80001076a594 which is weird (not a kernel address). The following 3 values are function arguments for bus_for_each_dev and they look correct.
From: Chan Kim <ckim@etri.re.kr>
Sent: Saturday, April 23, 2022 8:37 AM
To: 'Mulyadi Santosa' <mulyadi.santosa@gmail.com>
Cc: 'qemu-devel' <qemu-devel@nongnu.org>; 'kernelnewbies' <kernelnewbies@kernelnewbies.org>
Subject: RE: Backtrace stopped: previous frame identical to this frame (corrupt stack?) , even with fresh qemu and linux build
Hi, Mulyadi
Thank you for replying.
I found CONFIG_DEBUG_FRAME_POINTER, CONFIG_DEBUG_INFO are already set by default.
And I tried adding CONFIG_DEBUG_KERNEL, CONFIG_KGDB, CONFIG_GDB_SCRIPTS, CONFIG_STACKTRACE all to no avail.
Regards,
Chan
From: Mulyadi Santosa <mulyadi.santosa@gmail.com <mailto:mulyadi.santosa@gmail.com> >
Sent: Friday, April 22, 2022 11:53 PM
To: Chan Kim <ckim@etri.re.kr <mailto:ckim@etri.re.kr> >
Cc: qemu-devel <qemu-devel@nongnu.org <mailto:qemu-devel@nongnu.org> >; kernelnewbies <kernelnewbies@kernelnewbies.org <mailto:kernelnewbies@kernelnewbies.org> >
Subject: Re: Backtrace stopped: previous frame identical to this frame (corrupt stack?) , even with fresh qemu and linux build
On Fri, Apr 22, 2022 at 7:30 PM Chan Kim <ckim@etri.re.kr <mailto:ckim@etri.re.kr> > wrote:
Hello all,
Really strange thing happening here.. I can't see the full stack trace with
'bt' command in gdb.
So I tried with fresh linux-5.10.122 source and qemu-6.2.0 source and it's
happening too!
(it's happening when I do combinations with linux 5.10.0 and qemu-5.1.0. But
it's not happening with linux-5.4.21)
I would be grateful if somebody could check if this happens to other people
or just me.
1. download linux-5.1.122 tarball from https://www.kernel.org/
2. uncompress it and set env variable ARCH=arm64,
CROSS_COMPILE=aarch64-none-elf- , do "make defconfig" and "make -j`nproc`
Image"
3. download qemu-6.2.0 from https://www.qemu.org/
4. uncompress it and do "mkdir build" "cd build" "../configure
--target-list=aarch64-softmmu --enable-debug"
5. run qemu and wait for debugger to attach.
qemu-6.2.0/build/aarch64-softmmu/qemu-system-aarch64 -machine
virt,gic-version=max,secure=off,virtualization=true -cpu max -kernel
linux-5.10.112/arch/arm64/boot/Image -m 2G -nographic -netdev
user,id=vnet,hostfwd=:127.0.0.1:0-:22,tftp=/srv/tftp -device
virtio-net-pci,netdev=vnet -machine iommu=smmuv3 --append "root=/dev/ram
init=/init nokaslr earlycon ip=dhcp hugepages=16" -s -S
6. run debugger, do "aarch64-none-elf-gdb linux-6.10.112/vmlinux -x\
It has been long time since I compiled linux kernel but I guess, either you need to compile kernel with enabled frame pointer, and/or you need to enable debug symbol embedded into final kernel image. cmiiw
gdb_script"
(gdb_script content :
target remote :1234
layout src
b start_kernel
b __driver_attach
)
Now, in gdb, when you press 'c' twice, it'll stop at the first
__driver_attach. (first one stops at start_kernel).
When you are at __attach_driver, type 'bt'. See if you see the full function
stack trace.
This is what I see.
(gdb) bt
#0 __driver_attach (dev=0xffff000002582810, data=0xffff800011dc2358
<dummy_regulator_driver+40>)
at drivers/base/dd.c:1060
#1 0xffff8000107a3ed0 in bus_for_each_dev (bus=<optimized out>,
start=<optimized out>,
data=0xffff800011dc2358 <dummy_regulator_driver+40>,
fn=0xffff8000107a6f60 <__driver_attach>)
at drivers/base/bus.c:305
#2 0xd6d78000107a5c58 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
I used to see more thatn 20 stacks trace but strangely I see only two.
I can still see many stacks for linux-5.4.21 that I was working with in the
past.
Could anyone check if this happens to anyone?
I think if I add BLK_DEV_RAM and set initramfs.cpio.gz in the linux build,
the kernel will boot ok to the shell prompt.
Only the gdb can't show the stack levels.
My OS : ubuntu-20.04 5.13.0-35-generic
$ aarch64-none-elf-gdb --version
GNU gdb (GNU Toolchain for the A-profile Architecture 10.2-2020.11
(arm-10.16)) 10.1.90.20201028-git
Copyright (C) 2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
<http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Thank you.
Chan Kim
_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org <mailto:Kernelnewbies@kernelnewbies.org>
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
--
regards,
Mulyadi Santosa
Freelance Linux trainer and consultant
blog: the-hydra.blogspot.com <http://the-hydra.blogspot.com>
training: mulyaditraining.blogspot.com <http://mulyaditraining.blogspot.com>
[-- Attachment #1.2: Type: text/html, Size: 15182 bytes --]
[-- Attachment #2: Type: text/plain, Size: 170 bytes --]
_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
^ permalink raw reply [flat|nested] 12+ messages in thread
* RE: Backtrace stopped: previous frame identical to this frame (corrupt stack?) , even with fresh qemu and linux build
@ 2022-04-26 1:46 ` Chan Kim
0 siblings, 0 replies; 12+ messages in thread
From: Chan Kim @ 2022-04-26 1:46 UTC (permalink / raw)
To: 'qemu-devel', 'kernelnewbies'
[-- Attachment #1: Type: text/plain, Size: 6552 bytes --]
Hello,
I hope somebody could check this case. It’s easily reproducible for anybody working with qemu and arm64 linux.
I returned to this problem and made another observation.(showing the back-trace is really broken).
In another incident of breakpoint at function __driver_attach, (right after it stopped at the function)
(gdb) bt
#0 __driver_attach (dev=dev@entry=0xffff0000401d1810, data=data@entry=0xffff800011bbbbb8 <mxc_gpio_driver+40>) at drivers/base/dd.c:1046
#1 0xffff8000107684f8 in bus_for_each_dev (bus=0xffff800011cba910 <platform_bus_type>, start=0x0, data=0xffff800011bbbbb8 <mxc_gpio_driver+40>, fn=0xffff80001076b860 <__driver_attach>) at drivers/base/bus.c:307
#2 0xb8cd80001076a594 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
(gdb) x/5g $sp
0xffff800011dcbcc0: 0xffff800011dcbd20 0xb8cd80001076a594
0xffff800011dcbcd0: 0xffff80001076b860 0xffff800011bbbbb8
0xffff800011dcbce0: 0x0000000000000000
Because it's right after the pc reached the function __driver_attach, the sp is still not updated from previous function (bus_for_each_dev). And the first two values at the $sp are supposed to be the fp and lr of the previous function (see https://stackoverflow.com/questions/66098678/understanding-aarch64-assembly-function-call-how-is-stack-operated arm64 stores previous function's fp and lr at the bottom of new stack frame as it enters a function). The lr (link register, the address to return after this bus_for_each_dev function) is really 0xb8cd80001076a594 which is weird (not a kernel address). The following 3 values are function arguments for bus_for_each_dev and they look correct.
From: Chan Kim <ckim@etri.re.kr>
Sent: Saturday, April 23, 2022 8:37 AM
To: 'Mulyadi Santosa' <mulyadi.santosa@gmail.com>
Cc: 'qemu-devel' <qemu-devel@nongnu.org>; 'kernelnewbies' <kernelnewbies@kernelnewbies.org>
Subject: RE: Backtrace stopped: previous frame identical to this frame (corrupt stack?) , even with fresh qemu and linux build
Hi, Mulyadi
Thank you for replying.
I found CONFIG_DEBUG_FRAME_POINTER, CONFIG_DEBUG_INFO are already set by default.
And I tried adding CONFIG_DEBUG_KERNEL, CONFIG_KGDB, CONFIG_GDB_SCRIPTS, CONFIG_STACKTRACE all to no avail.
Regards,
Chan
From: Mulyadi Santosa <mulyadi.santosa@gmail.com <mailto:mulyadi.santosa@gmail.com> >
Sent: Friday, April 22, 2022 11:53 PM
To: Chan Kim <ckim@etri.re.kr <mailto:ckim@etri.re.kr> >
Cc: qemu-devel <qemu-devel@nongnu.org <mailto:qemu-devel@nongnu.org> >; kernelnewbies <kernelnewbies@kernelnewbies.org <mailto:kernelnewbies@kernelnewbies.org> >
Subject: Re: Backtrace stopped: previous frame identical to this frame (corrupt stack?) , even with fresh qemu and linux build
On Fri, Apr 22, 2022 at 7:30 PM Chan Kim <ckim@etri.re.kr <mailto:ckim@etri.re.kr> > wrote:
Hello all,
Really strange thing happening here.. I can't see the full stack trace with
'bt' command in gdb.
So I tried with fresh linux-5.10.122 source and qemu-6.2.0 source and it's
happening too!
(it's happening when I do combinations with linux 5.10.0 and qemu-5.1.0. But
it's not happening with linux-5.4.21)
I would be grateful if somebody could check if this happens to other people
or just me.
1. download linux-5.1.122 tarball from https://www.kernel.org/
2. uncompress it and set env variable ARCH=arm64,
CROSS_COMPILE=aarch64-none-elf- , do "make defconfig" and "make -j`nproc`
Image"
3. download qemu-6.2.0 from https://www.qemu.org/
4. uncompress it and do "mkdir build" "cd build" "../configure
--target-list=aarch64-softmmu --enable-debug"
5. run qemu and wait for debugger to attach.
qemu-6.2.0/build/aarch64-softmmu/qemu-system-aarch64 -machine
virt,gic-version=max,secure=off,virtualization=true -cpu max -kernel
linux-5.10.112/arch/arm64/boot/Image -m 2G -nographic -netdev
user,id=vnet,hostfwd=:127.0.0.1:0-:22,tftp=/srv/tftp -device
virtio-net-pci,netdev=vnet -machine iommu=smmuv3 --append "root=/dev/ram
init=/init nokaslr earlycon ip=dhcp hugepages=16" -s -S
6. run debugger, do "aarch64-none-elf-gdb linux-6.10.112/vmlinux -x\
It has been long time since I compiled linux kernel but I guess, either you need to compile kernel with enabled frame pointer, and/or you need to enable debug symbol embedded into final kernel image. cmiiw
gdb_script"
(gdb_script content :
target remote :1234
layout src
b start_kernel
b __driver_attach
)
Now, in gdb, when you press 'c' twice, it'll stop at the first
__driver_attach. (first one stops at start_kernel).
When you are at __attach_driver, type 'bt'. See if you see the full function
stack trace.
This is what I see.
(gdb) bt
#0 __driver_attach (dev=0xffff000002582810, data=0xffff800011dc2358
<dummy_regulator_driver+40>)
at drivers/base/dd.c:1060
#1 0xffff8000107a3ed0 in bus_for_each_dev (bus=<optimized out>,
start=<optimized out>,
data=0xffff800011dc2358 <dummy_regulator_driver+40>,
fn=0xffff8000107a6f60 <__driver_attach>)
at drivers/base/bus.c:305
#2 0xd6d78000107a5c58 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
I used to see more thatn 20 stacks trace but strangely I see only two.
I can still see many stacks for linux-5.4.21 that I was working with in the
past.
Could anyone check if this happens to anyone?
I think if I add BLK_DEV_RAM and set initramfs.cpio.gz in the linux build,
the kernel will boot ok to the shell prompt.
Only the gdb can't show the stack levels.
My OS : ubuntu-20.04 5.13.0-35-generic
$ aarch64-none-elf-gdb --version
GNU gdb (GNU Toolchain for the A-profile Architecture 10.2-2020.11
(arm-10.16)) 10.1.90.20201028-git
Copyright (C) 2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
<http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Thank you.
Chan Kim
_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org <mailto:Kernelnewbies@kernelnewbies.org>
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
--
regards,
Mulyadi Santosa
Freelance Linux trainer and consultant
blog: the-hydra.blogspot.com <http://the-hydra.blogspot.com>
training: mulyaditraining.blogspot.com <http://mulyaditraining.blogspot.com>
[-- Attachment #2: Type: text/html, Size: 15182 bytes --]
^ permalink raw reply [flat|nested] 12+ messages in thread
* RE: Backtrace stopped: previous frame identical to this frame (corrupt stack?) , even with fresh qemu and linux build
2022-04-22 14:52 ` Mulyadi Santosa
@ 2022-04-27 9:05 ` Chan Kim
-1 siblings, 0 replies; 12+ messages in thread
From: Chan Kim @ 2022-04-27 9:05 UTC (permalink / raw)
To: 'qemu-devel', 'kernelnewbies'
[-- Attachment #1.1: Type: text/plain, Size: 7545 bytes --]
Hi, all
I just found that when the linux kernel saves the current x29 and x30 at the new stack bottom, (it usually does that when entering a function)
The stored x30 value (lr register) has it top 16bits altered to some strange value. So if I fix those top 16bits to 0xffff in the stack, and if I repeat fixing this stored x30 values, I can see more backtrace information. I filed a bug report to bug.linaro.org. This strange thing doesn’t happen with linux version < 5.10 to me so it doesn’t look like qemu bug.
Chan Kim
From: Chan Kim <ckim@etri.re.kr>
Sent: Tuesday, April 26, 2022 10:47 AM
To: 'qemu-devel' <qemu-devel@nongnu.org>; 'kernelnewbies' <kernelnewbies@kernelnewbies.org>
Subject: RE: Backtrace stopped: previous frame identical to this frame (corrupt stack?) , even with fresh qemu and linux build
Hello,
I hope somebody could check this case. It’s easily reproducible for anybody working with qemu and arm64 linux.
I returned to this problem and made another observation.(showing the back-trace is really broken).
In another incident of breakpoint at function __driver_attach, (right after it stopped at the function)
(gdb) bt
#0 __driver_attach (dev=dev@entry=0xffff0000401d1810, data=data@entry=0xffff800011bbbbb8 <mxc_gpio_driver+40>) at drivers/base/dd.c:1046
#1 0xffff8000107684f8 in bus_for_each_dev (bus=0xffff800011cba910 <platform_bus_type>, start=0x0, data=0xffff800011bbbbb8 <mxc_gpio_driver+40>, fn=0xffff80001076b860 <__driver_attach>) at drivers/base/bus.c:307
#2 0xb8cd80001076a594 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
(gdb) x/5g $sp
0xffff800011dcbcc0: 0xffff800011dcbd20 0xb8cd80001076a594
0xffff800011dcbcd0: 0xffff80001076b860 0xffff800011bbbbb8
0xffff800011dcbce0: 0x0000000000000000
Because it's right after the pc reached the function __driver_attach, the sp is still not updated from previous function (bus_for_each_dev). And the first two values at the $sp are supposed to be the fp and lr of the previous function (see https://stackoverflow.com/questions/66098678/understanding-aarch64-assembly-function-call-how-is-stack-operated arm64 stores previous function's fp and lr at the bottom of new stack frame as it enters a function). The lr (link register, the address to return after this bus_for_each_dev function) is really 0xb8cd80001076a594 which is weird (not a kernel address). The following 3 values are function arguments for bus_for_each_dev and they look correct.
From: Chan Kim <ckim@etri.re.kr <mailto:ckim@etri.re.kr> >
Sent: Saturday, April 23, 2022 8:37 AM
To: 'Mulyadi Santosa' <mulyadi.santosa@gmail.com <mailto:mulyadi.santosa@gmail.com> >
Cc: 'qemu-devel' <qemu-devel@nongnu.org <mailto:qemu-devel@nongnu.org> >; 'kernelnewbies' <kernelnewbies@kernelnewbies.org <mailto:kernelnewbies@kernelnewbies.org> >
Subject: RE: Backtrace stopped: previous frame identical to this frame (corrupt stack?) , even with fresh qemu and linux build
Hi, Mulyadi
Thank you for replying.
I found CONFIG_DEBUG_FRAME_POINTER, CONFIG_DEBUG_INFO are already set by default.
And I tried adding CONFIG_DEBUG_KERNEL, CONFIG_KGDB, CONFIG_GDB_SCRIPTS, CONFIG_STACKTRACE all to no avail.
Regards,
Chan
From: Mulyadi Santosa <mulyadi.santosa@gmail.com <mailto:mulyadi.santosa@gmail.com> >
Sent: Friday, April 22, 2022 11:53 PM
To: Chan Kim <ckim@etri.re.kr <mailto:ckim@etri.re.kr> >
Cc: qemu-devel <qemu-devel@nongnu.org <mailto:qemu-devel@nongnu.org> >; kernelnewbies <kernelnewbies@kernelnewbies.org <mailto:kernelnewbies@kernelnewbies.org> >
Subject: Re: Backtrace stopped: previous frame identical to this frame (corrupt stack?) , even with fresh qemu and linux build
On Fri, Apr 22, 2022 at 7:30 PM Chan Kim <ckim@etri.re.kr <mailto:ckim@etri.re.kr> > wrote:
Hello all,
Really strange thing happening here.. I can't see the full stack trace with
'bt' command in gdb.
So I tried with fresh linux-5.10.122 source and qemu-6.2.0 source and it's
happening too!
(it's happening when I do combinations with linux 5.10.0 and qemu-5.1.0. But
it's not happening with linux-5.4.21)
I would be grateful if somebody could check if this happens to other people
or just me.
1. download linux-5.1.122 tarball from https://www.kernel.org/
2. uncompress it and set env variable ARCH=arm64,
CROSS_COMPILE=aarch64-none-elf- , do "make defconfig" and "make -j`nproc`
Image"
3. download qemu-6.2.0 from https://www.qemu.org/
4. uncompress it and do "mkdir build" "cd build" "../configure
--target-list=aarch64-softmmu --enable-debug"
5. run qemu and wait for debugger to attach.
qemu-6.2.0/build/aarch64-softmmu/qemu-system-aarch64 -machine
virt,gic-version=max,secure=off,virtualization=true -cpu max -kernel
linux-5.10.112/arch/arm64/boot/Image -m 2G -nographic -netdev
user,id=vnet,hostfwd=:127.0.0.1:0-:22,tftp=/srv/tftp -device
virtio-net-pci,netdev=vnet -machine iommu=smmuv3 --append "root=/dev/ram
init=/init nokaslr earlycon ip=dhcp hugepages=16" -s -S
6. run debugger, do "aarch64-none-elf-gdb linux-6.10.112/vmlinux -x\
It has been long time since I compiled linux kernel but I guess, either you need to compile kernel with enabled frame pointer, and/or you need to enable debug symbol embedded into final kernel image. cmiiw
gdb_script"
(gdb_script content :
target remote :1234
layout src
b start_kernel
b __driver_attach
)
Now, in gdb, when you press 'c' twice, it'll stop at the first
__driver_attach. (first one stops at start_kernel).
When you are at __attach_driver, type 'bt'. See if you see the full function
stack trace.
This is what I see.
(gdb) bt
#0 __driver_attach (dev=0xffff000002582810, data=0xffff800011dc2358
<dummy_regulator_driver+40>)
at drivers/base/dd.c:1060
#1 0xffff8000107a3ed0 in bus_for_each_dev (bus=<optimized out>,
start=<optimized out>,
data=0xffff800011dc2358 <dummy_regulator_driver+40>,
fn=0xffff8000107a6f60 <__driver_attach>)
at drivers/base/bus.c:305
#2 0xd6d78000107a5c58 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
I used to see more thatn 20 stacks trace but strangely I see only two.
I can still see many stacks for linux-5.4.21 that I was working with in the
past.
Could anyone check if this happens to anyone?
I think if I add BLK_DEV_RAM and set initramfs.cpio.gz in the linux build,
the kernel will boot ok to the shell prompt.
Only the gdb can't show the stack levels.
My OS : ubuntu-20.04 5.13.0-35-generic
$ aarch64-none-elf-gdb --version
GNU gdb (GNU Toolchain for the A-profile Architecture 10.2-2020.11
(arm-10.16)) 10.1.90.20201028-git
Copyright (C) 2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
<http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Thank you.
Chan Kim
_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org <mailto:Kernelnewbies@kernelnewbies.org>
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
--
regards,
Mulyadi Santosa
Freelance Linux trainer and consultant
blog: the-hydra.blogspot.com <http://the-hydra.blogspot.com>
training: mulyaditraining.blogspot.com <http://mulyaditraining.blogspot.com>
[-- Attachment #1.2: Type: text/html, Size: 18106 bytes --]
[-- Attachment #2: Type: text/plain, Size: 170 bytes --]
_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
^ permalink raw reply [flat|nested] 12+ messages in thread
* RE: Backtrace stopped: previous frame identical to this frame (corrupt stack?) , even with fresh qemu and linux build
@ 2022-04-27 9:05 ` Chan Kim
0 siblings, 0 replies; 12+ messages in thread
From: Chan Kim @ 2022-04-27 9:05 UTC (permalink / raw)
To: 'qemu-devel', 'kernelnewbies'
[-- Attachment #1: Type: text/plain, Size: 7545 bytes --]
Hi, all
I just found that when the linux kernel saves the current x29 and x30 at the new stack bottom, (it usually does that when entering a function)
The stored x30 value (lr register) has it top 16bits altered to some strange value. So if I fix those top 16bits to 0xffff in the stack, and if I repeat fixing this stored x30 values, I can see more backtrace information. I filed a bug report to bug.linaro.org. This strange thing doesn’t happen with linux version < 5.10 to me so it doesn’t look like qemu bug.
Chan Kim
From: Chan Kim <ckim@etri.re.kr>
Sent: Tuesday, April 26, 2022 10:47 AM
To: 'qemu-devel' <qemu-devel@nongnu.org>; 'kernelnewbies' <kernelnewbies@kernelnewbies.org>
Subject: RE: Backtrace stopped: previous frame identical to this frame (corrupt stack?) , even with fresh qemu and linux build
Hello,
I hope somebody could check this case. It’s easily reproducible for anybody working with qemu and arm64 linux.
I returned to this problem and made another observation.(showing the back-trace is really broken).
In another incident of breakpoint at function __driver_attach, (right after it stopped at the function)
(gdb) bt
#0 __driver_attach (dev=dev@entry=0xffff0000401d1810, data=data@entry=0xffff800011bbbbb8 <mxc_gpio_driver+40>) at drivers/base/dd.c:1046
#1 0xffff8000107684f8 in bus_for_each_dev (bus=0xffff800011cba910 <platform_bus_type>, start=0x0, data=0xffff800011bbbbb8 <mxc_gpio_driver+40>, fn=0xffff80001076b860 <__driver_attach>) at drivers/base/bus.c:307
#2 0xb8cd80001076a594 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
(gdb) x/5g $sp
0xffff800011dcbcc0: 0xffff800011dcbd20 0xb8cd80001076a594
0xffff800011dcbcd0: 0xffff80001076b860 0xffff800011bbbbb8
0xffff800011dcbce0: 0x0000000000000000
Because it's right after the pc reached the function __driver_attach, the sp is still not updated from previous function (bus_for_each_dev). And the first two values at the $sp are supposed to be the fp and lr of the previous function (see https://stackoverflow.com/questions/66098678/understanding-aarch64-assembly-function-call-how-is-stack-operated arm64 stores previous function's fp and lr at the bottom of new stack frame as it enters a function). The lr (link register, the address to return after this bus_for_each_dev function) is really 0xb8cd80001076a594 which is weird (not a kernel address). The following 3 values are function arguments for bus_for_each_dev and they look correct.
From: Chan Kim <ckim@etri.re.kr <mailto:ckim@etri.re.kr> >
Sent: Saturday, April 23, 2022 8:37 AM
To: 'Mulyadi Santosa' <mulyadi.santosa@gmail.com <mailto:mulyadi.santosa@gmail.com> >
Cc: 'qemu-devel' <qemu-devel@nongnu.org <mailto:qemu-devel@nongnu.org> >; 'kernelnewbies' <kernelnewbies@kernelnewbies.org <mailto:kernelnewbies@kernelnewbies.org> >
Subject: RE: Backtrace stopped: previous frame identical to this frame (corrupt stack?) , even with fresh qemu and linux build
Hi, Mulyadi
Thank you for replying.
I found CONFIG_DEBUG_FRAME_POINTER, CONFIG_DEBUG_INFO are already set by default.
And I tried adding CONFIG_DEBUG_KERNEL, CONFIG_KGDB, CONFIG_GDB_SCRIPTS, CONFIG_STACKTRACE all to no avail.
Regards,
Chan
From: Mulyadi Santosa <mulyadi.santosa@gmail.com <mailto:mulyadi.santosa@gmail.com> >
Sent: Friday, April 22, 2022 11:53 PM
To: Chan Kim <ckim@etri.re.kr <mailto:ckim@etri.re.kr> >
Cc: qemu-devel <qemu-devel@nongnu.org <mailto:qemu-devel@nongnu.org> >; kernelnewbies <kernelnewbies@kernelnewbies.org <mailto:kernelnewbies@kernelnewbies.org> >
Subject: Re: Backtrace stopped: previous frame identical to this frame (corrupt stack?) , even with fresh qemu and linux build
On Fri, Apr 22, 2022 at 7:30 PM Chan Kim <ckim@etri.re.kr <mailto:ckim@etri.re.kr> > wrote:
Hello all,
Really strange thing happening here.. I can't see the full stack trace with
'bt' command in gdb.
So I tried with fresh linux-5.10.122 source and qemu-6.2.0 source and it's
happening too!
(it's happening when I do combinations with linux 5.10.0 and qemu-5.1.0. But
it's not happening with linux-5.4.21)
I would be grateful if somebody could check if this happens to other people
or just me.
1. download linux-5.1.122 tarball from https://www.kernel.org/
2. uncompress it and set env variable ARCH=arm64,
CROSS_COMPILE=aarch64-none-elf- , do "make defconfig" and "make -j`nproc`
Image"
3. download qemu-6.2.0 from https://www.qemu.org/
4. uncompress it and do "mkdir build" "cd build" "../configure
--target-list=aarch64-softmmu --enable-debug"
5. run qemu and wait for debugger to attach.
qemu-6.2.0/build/aarch64-softmmu/qemu-system-aarch64 -machine
virt,gic-version=max,secure=off,virtualization=true -cpu max -kernel
linux-5.10.112/arch/arm64/boot/Image -m 2G -nographic -netdev
user,id=vnet,hostfwd=:127.0.0.1:0-:22,tftp=/srv/tftp -device
virtio-net-pci,netdev=vnet -machine iommu=smmuv3 --append "root=/dev/ram
init=/init nokaslr earlycon ip=dhcp hugepages=16" -s -S
6. run debugger, do "aarch64-none-elf-gdb linux-6.10.112/vmlinux -x\
It has been long time since I compiled linux kernel but I guess, either you need to compile kernel with enabled frame pointer, and/or you need to enable debug symbol embedded into final kernel image. cmiiw
gdb_script"
(gdb_script content :
target remote :1234
layout src
b start_kernel
b __driver_attach
)
Now, in gdb, when you press 'c' twice, it'll stop at the first
__driver_attach. (first one stops at start_kernel).
When you are at __attach_driver, type 'bt'. See if you see the full function
stack trace.
This is what I see.
(gdb) bt
#0 __driver_attach (dev=0xffff000002582810, data=0xffff800011dc2358
<dummy_regulator_driver+40>)
at drivers/base/dd.c:1060
#1 0xffff8000107a3ed0 in bus_for_each_dev (bus=<optimized out>,
start=<optimized out>,
data=0xffff800011dc2358 <dummy_regulator_driver+40>,
fn=0xffff8000107a6f60 <__driver_attach>)
at drivers/base/bus.c:305
#2 0xd6d78000107a5c58 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
I used to see more thatn 20 stacks trace but strangely I see only two.
I can still see many stacks for linux-5.4.21 that I was working with in the
past.
Could anyone check if this happens to anyone?
I think if I add BLK_DEV_RAM and set initramfs.cpio.gz in the linux build,
the kernel will boot ok to the shell prompt.
Only the gdb can't show the stack levels.
My OS : ubuntu-20.04 5.13.0-35-generic
$ aarch64-none-elf-gdb --version
GNU gdb (GNU Toolchain for the A-profile Architecture 10.2-2020.11
(arm-10.16)) 10.1.90.20201028-git
Copyright (C) 2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
<http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Thank you.
Chan Kim
_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org <mailto:Kernelnewbies@kernelnewbies.org>
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
--
regards,
Mulyadi Santosa
Freelance Linux trainer and consultant
blog: the-hydra.blogspot.com <http://the-hydra.blogspot.com>
training: mulyaditraining.blogspot.com <http://mulyaditraining.blogspot.com>
[-- Attachment #2: Type: text/html, Size: 18106 bytes --]
^ permalink raw reply [flat|nested] 12+ messages in thread
* RE: Backtrace stopped: previous frame identical to this frame (corrupt stack?) , even with fresh qemu and linux build
2022-04-22 14:52 ` Mulyadi Santosa
@ 2022-04-27 9:19 ` Chan Kim
-1 siblings, 0 replies; 12+ messages in thread
From: Chan Kim @ 2022-04-27 9:19 UTC (permalink / raw)
To: 'qemu-devel', 'kernelnewbies'
[-- Attachment #1.1: Type: text/plain, Size: 8207 bytes --]
Hi folks,
Sorry for many emails.
I just found out by turning CONFIG_ARM64_PTR_AUTH off in armv8.3 when building linux, I can avoid this problem.
(I noticed the instruction ‘pacia’ in the function assembly code)
Thank you.
Chan Kim
From: Chan Kim <ckim@etri.re.kr>
Sent: Wednesday, April 27, 2022 6:05 PM
To: 'qemu-devel' <qemu-devel@nongnu.org>; 'kernelnewbies' <kernelnewbies@kernelnewbies.org>
Subject: RE: Backtrace stopped: previous frame identical to this frame (corrupt stack?) , even with fresh qemu and linux build
Hi, all
I just found that when the linux kernel saves the current x29 and x30 at the new stack bottom, (it usually does that when entering a function)
The stored x30 value (lr register) has it top 16bits altered to some strange value. So if I fix those top 16bits to 0xffff in the stack, and if I repeat fixing this stored x30 values, I can see more backtrace information. I filed a bug report to bug.linaro.org. This strange thing doesn’t happen with linux version < 5.10 to me so it doesn’t look like qemu bug.
Chan Kim
From: Chan Kim <ckim@etri.re.kr <mailto:ckim@etri.re.kr> >
Sent: Tuesday, April 26, 2022 10:47 AM
To: 'qemu-devel' <qemu-devel@nongnu.org <mailto:qemu-devel@nongnu.org> >; 'kernelnewbies' <kernelnewbies@kernelnewbies.org <mailto:kernelnewbies@kernelnewbies.org> >
Subject: RE: Backtrace stopped: previous frame identical to this frame (corrupt stack?) , even with fresh qemu and linux build
Hello,
I hope somebody could check this case. It’s easily reproducible for anybody working with qemu and arm64 linux.
I returned to this problem and made another observation.(showing the back-trace is really broken).
In another incident of breakpoint at function __driver_attach, (right after it stopped at the function)
(gdb) bt
#0 __driver_attach (dev=dev@entry=0xffff0000401d1810, data=data@entry=0xffff800011bbbbb8 <mxc_gpio_driver+40>) at drivers/base/dd.c:1046
#1 0xffff8000107684f8 in bus_for_each_dev (bus=0xffff800011cba910 <platform_bus_type>, start=0x0, data=0xffff800011bbbbb8 <mxc_gpio_driver+40>, fn=0xffff80001076b860 <__driver_attach>) at drivers/base/bus.c:307
#2 0xb8cd80001076a594 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
(gdb) x/5g $sp
0xffff800011dcbcc0: 0xffff800011dcbd20 0xb8cd80001076a594
0xffff800011dcbcd0: 0xffff80001076b860 0xffff800011bbbbb8
0xffff800011dcbce0: 0x0000000000000000
Because it's right after the pc reached the function __driver_attach, the sp is still not updated from previous function (bus_for_each_dev). And the first two values at the $sp are supposed to be the fp and lr of the previous function (see https://stackoverflow.com/questions/66098678/understanding-aarch64-assembly-function-call-how-is-stack-operated arm64 stores previous function's fp and lr at the bottom of new stack frame as it enters a function). The lr (link register, the address to return after this bus_for_each_dev function) is really 0xb8cd80001076a594 which is weird (not a kernel address). The following 3 values are function arguments for bus_for_each_dev and they look correct.
From: Chan Kim <ckim@etri.re.kr <mailto:ckim@etri.re.kr> >
Sent: Saturday, April 23, 2022 8:37 AM
To: 'Mulyadi Santosa' <mulyadi.santosa@gmail.com <mailto:mulyadi.santosa@gmail.com> >
Cc: 'qemu-devel' <qemu-devel@nongnu.org <mailto:qemu-devel@nongnu.org> >; 'kernelnewbies' <kernelnewbies@kernelnewbies.org <mailto:kernelnewbies@kernelnewbies.org> >
Subject: RE: Backtrace stopped: previous frame identical to this frame (corrupt stack?) , even with fresh qemu and linux build
Hi, Mulyadi
Thank you for replying.
I found CONFIG_DEBUG_FRAME_POINTER, CONFIG_DEBUG_INFO are already set by default.
And I tried adding CONFIG_DEBUG_KERNEL, CONFIG_KGDB, CONFIG_GDB_SCRIPTS, CONFIG_STACKTRACE all to no avail.
Regards,
Chan
From: Mulyadi Santosa <mulyadi.santosa@gmail.com <mailto:mulyadi.santosa@gmail.com> >
Sent: Friday, April 22, 2022 11:53 PM
To: Chan Kim <ckim@etri.re.kr <mailto:ckim@etri.re.kr> >
Cc: qemu-devel <qemu-devel@nongnu.org <mailto:qemu-devel@nongnu.org> >; kernelnewbies <kernelnewbies@kernelnewbies.org <mailto:kernelnewbies@kernelnewbies.org> >
Subject: Re: Backtrace stopped: previous frame identical to this frame (corrupt stack?) , even with fresh qemu and linux build
On Fri, Apr 22, 2022 at 7:30 PM Chan Kim <ckim@etri.re.kr <mailto:ckim@etri.re.kr> > wrote:
Hello all,
Really strange thing happening here.. I can't see the full stack trace with
'bt' command in gdb.
So I tried with fresh linux-5.10.122 source and qemu-6.2.0 source and it's
happening too!
(it's happening when I do combinations with linux 5.10.0 and qemu-5.1.0. But
it's not happening with linux-5.4.21)
I would be grateful if somebody could check if this happens to other people
or just me.
1. download linux-5.1.122 tarball from https://www.kernel.org/
2. uncompress it and set env variable ARCH=arm64,
CROSS_COMPILE=aarch64-none-elf- , do "make defconfig" and "make -j`nproc`
Image"
3. download qemu-6.2.0 from https://www.qemu.org/
4. uncompress it and do "mkdir build" "cd build" "../configure
--target-list=aarch64-softmmu --enable-debug"
5. run qemu and wait for debugger to attach.
qemu-6.2.0/build/aarch64-softmmu/qemu-system-aarch64 -machine
virt,gic-version=max,secure=off,virtualization=true -cpu max -kernel
linux-5.10.112/arch/arm64/boot/Image -m 2G -nographic -netdev
user,id=vnet,hostfwd=:127.0.0.1:0-:22,tftp=/srv/tftp -device
virtio-net-pci,netdev=vnet -machine iommu=smmuv3 --append "root=/dev/ram
init=/init nokaslr earlycon ip=dhcp hugepages=16" -s -S
6. run debugger, do "aarch64-none-elf-gdb linux-6.10.112/vmlinux -x\
It has been long time since I compiled linux kernel but I guess, either you need to compile kernel with enabled frame pointer, and/or you need to enable debug symbol embedded into final kernel image. cmiiw
gdb_script"
(gdb_script content :
target remote :1234
layout src
b start_kernel
b __driver_attach
)
Now, in gdb, when you press 'c' twice, it'll stop at the first
__driver_attach. (first one stops at start_kernel).
When you are at __attach_driver, type 'bt'. See if you see the full function
stack trace.
This is what I see.
(gdb) bt
#0 __driver_attach (dev=0xffff000002582810, data=0xffff800011dc2358
<dummy_regulator_driver+40>)
at drivers/base/dd.c:1060
#1 0xffff8000107a3ed0 in bus_for_each_dev (bus=<optimized out>,
start=<optimized out>,
data=0xffff800011dc2358 <dummy_regulator_driver+40>,
fn=0xffff8000107a6f60 <__driver_attach>)
at drivers/base/bus.c:305
#2 0xd6d78000107a5c58 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
I used to see more thatn 20 stacks trace but strangely I see only two.
I can still see many stacks for linux-5.4.21 that I was working with in the
past.
Could anyone check if this happens to anyone?
I think if I add BLK_DEV_RAM and set initramfs.cpio.gz in the linux build,
the kernel will boot ok to the shell prompt.
Only the gdb can't show the stack levels.
My OS : ubuntu-20.04 5.13.0-35-generic
$ aarch64-none-elf-gdb --version
GNU gdb (GNU Toolchain for the A-profile Architecture 10.2-2020.11
(arm-10.16)) 10.1.90.20201028-git
Copyright (C) 2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
<http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Thank you.
Chan Kim
_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org <mailto:Kernelnewbies@kernelnewbies.org>
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
--
regards,
Mulyadi Santosa
Freelance Linux trainer and consultant
blog: the-hydra.blogspot.com <http://the-hydra.blogspot.com>
training: mulyaditraining.blogspot.com <http://mulyaditraining.blogspot.com>
[-- Attachment #1.2: Type: text/html, Size: 20482 bytes --]
[-- Attachment #2: Type: text/plain, Size: 170 bytes --]
_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
^ permalink raw reply [flat|nested] 12+ messages in thread
* RE: Backtrace stopped: previous frame identical to this frame (corrupt stack?) , even with fresh qemu and linux build
@ 2022-04-27 9:19 ` Chan Kim
0 siblings, 0 replies; 12+ messages in thread
From: Chan Kim @ 2022-04-27 9:19 UTC (permalink / raw)
To: 'qemu-devel', 'kernelnewbies'
[-- Attachment #1: Type: text/plain, Size: 8207 bytes --]
Hi folks,
Sorry for many emails.
I just found out by turning CONFIG_ARM64_PTR_AUTH off in armv8.3 when building linux, I can avoid this problem.
(I noticed the instruction ‘pacia’ in the function assembly code)
Thank you.
Chan Kim
From: Chan Kim <ckim@etri.re.kr>
Sent: Wednesday, April 27, 2022 6:05 PM
To: 'qemu-devel' <qemu-devel@nongnu.org>; 'kernelnewbies' <kernelnewbies@kernelnewbies.org>
Subject: RE: Backtrace stopped: previous frame identical to this frame (corrupt stack?) , even with fresh qemu and linux build
Hi, all
I just found that when the linux kernel saves the current x29 and x30 at the new stack bottom, (it usually does that when entering a function)
The stored x30 value (lr register) has it top 16bits altered to some strange value. So if I fix those top 16bits to 0xffff in the stack, and if I repeat fixing this stored x30 values, I can see more backtrace information. I filed a bug report to bug.linaro.org. This strange thing doesn’t happen with linux version < 5.10 to me so it doesn’t look like qemu bug.
Chan Kim
From: Chan Kim <ckim@etri.re.kr <mailto:ckim@etri.re.kr> >
Sent: Tuesday, April 26, 2022 10:47 AM
To: 'qemu-devel' <qemu-devel@nongnu.org <mailto:qemu-devel@nongnu.org> >; 'kernelnewbies' <kernelnewbies@kernelnewbies.org <mailto:kernelnewbies@kernelnewbies.org> >
Subject: RE: Backtrace stopped: previous frame identical to this frame (corrupt stack?) , even with fresh qemu and linux build
Hello,
I hope somebody could check this case. It’s easily reproducible for anybody working with qemu and arm64 linux.
I returned to this problem and made another observation.(showing the back-trace is really broken).
In another incident of breakpoint at function __driver_attach, (right after it stopped at the function)
(gdb) bt
#0 __driver_attach (dev=dev@entry=0xffff0000401d1810, data=data@entry=0xffff800011bbbbb8 <mxc_gpio_driver+40>) at drivers/base/dd.c:1046
#1 0xffff8000107684f8 in bus_for_each_dev (bus=0xffff800011cba910 <platform_bus_type>, start=0x0, data=0xffff800011bbbbb8 <mxc_gpio_driver+40>, fn=0xffff80001076b860 <__driver_attach>) at drivers/base/bus.c:307
#2 0xb8cd80001076a594 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
(gdb) x/5g $sp
0xffff800011dcbcc0: 0xffff800011dcbd20 0xb8cd80001076a594
0xffff800011dcbcd0: 0xffff80001076b860 0xffff800011bbbbb8
0xffff800011dcbce0: 0x0000000000000000
Because it's right after the pc reached the function __driver_attach, the sp is still not updated from previous function (bus_for_each_dev). And the first two values at the $sp are supposed to be the fp and lr of the previous function (see https://stackoverflow.com/questions/66098678/understanding-aarch64-assembly-function-call-how-is-stack-operated arm64 stores previous function's fp and lr at the bottom of new stack frame as it enters a function). The lr (link register, the address to return after this bus_for_each_dev function) is really 0xb8cd80001076a594 which is weird (not a kernel address). The following 3 values are function arguments for bus_for_each_dev and they look correct.
From: Chan Kim <ckim@etri.re.kr <mailto:ckim@etri.re.kr> >
Sent: Saturday, April 23, 2022 8:37 AM
To: 'Mulyadi Santosa' <mulyadi.santosa@gmail.com <mailto:mulyadi.santosa@gmail.com> >
Cc: 'qemu-devel' <qemu-devel@nongnu.org <mailto:qemu-devel@nongnu.org> >; 'kernelnewbies' <kernelnewbies@kernelnewbies.org <mailto:kernelnewbies@kernelnewbies.org> >
Subject: RE: Backtrace stopped: previous frame identical to this frame (corrupt stack?) , even with fresh qemu and linux build
Hi, Mulyadi
Thank you for replying.
I found CONFIG_DEBUG_FRAME_POINTER, CONFIG_DEBUG_INFO are already set by default.
And I tried adding CONFIG_DEBUG_KERNEL, CONFIG_KGDB, CONFIG_GDB_SCRIPTS, CONFIG_STACKTRACE all to no avail.
Regards,
Chan
From: Mulyadi Santosa <mulyadi.santosa@gmail.com <mailto:mulyadi.santosa@gmail.com> >
Sent: Friday, April 22, 2022 11:53 PM
To: Chan Kim <ckim@etri.re.kr <mailto:ckim@etri.re.kr> >
Cc: qemu-devel <qemu-devel@nongnu.org <mailto:qemu-devel@nongnu.org> >; kernelnewbies <kernelnewbies@kernelnewbies.org <mailto:kernelnewbies@kernelnewbies.org> >
Subject: Re: Backtrace stopped: previous frame identical to this frame (corrupt stack?) , even with fresh qemu and linux build
On Fri, Apr 22, 2022 at 7:30 PM Chan Kim <ckim@etri.re.kr <mailto:ckim@etri.re.kr> > wrote:
Hello all,
Really strange thing happening here.. I can't see the full stack trace with
'bt' command in gdb.
So I tried with fresh linux-5.10.122 source and qemu-6.2.0 source and it's
happening too!
(it's happening when I do combinations with linux 5.10.0 and qemu-5.1.0. But
it's not happening with linux-5.4.21)
I would be grateful if somebody could check if this happens to other people
or just me.
1. download linux-5.1.122 tarball from https://www.kernel.org/
2. uncompress it and set env variable ARCH=arm64,
CROSS_COMPILE=aarch64-none-elf- , do "make defconfig" and "make -j`nproc`
Image"
3. download qemu-6.2.0 from https://www.qemu.org/
4. uncompress it and do "mkdir build" "cd build" "../configure
--target-list=aarch64-softmmu --enable-debug"
5. run qemu and wait for debugger to attach.
qemu-6.2.0/build/aarch64-softmmu/qemu-system-aarch64 -machine
virt,gic-version=max,secure=off,virtualization=true -cpu max -kernel
linux-5.10.112/arch/arm64/boot/Image -m 2G -nographic -netdev
user,id=vnet,hostfwd=:127.0.0.1:0-:22,tftp=/srv/tftp -device
virtio-net-pci,netdev=vnet -machine iommu=smmuv3 --append "root=/dev/ram
init=/init nokaslr earlycon ip=dhcp hugepages=16" -s -S
6. run debugger, do "aarch64-none-elf-gdb linux-6.10.112/vmlinux -x\
It has been long time since I compiled linux kernel but I guess, either you need to compile kernel with enabled frame pointer, and/or you need to enable debug symbol embedded into final kernel image. cmiiw
gdb_script"
(gdb_script content :
target remote :1234
layout src
b start_kernel
b __driver_attach
)
Now, in gdb, when you press 'c' twice, it'll stop at the first
__driver_attach. (first one stops at start_kernel).
When you are at __attach_driver, type 'bt'. See if you see the full function
stack trace.
This is what I see.
(gdb) bt
#0 __driver_attach (dev=0xffff000002582810, data=0xffff800011dc2358
<dummy_regulator_driver+40>)
at drivers/base/dd.c:1060
#1 0xffff8000107a3ed0 in bus_for_each_dev (bus=<optimized out>,
start=<optimized out>,
data=0xffff800011dc2358 <dummy_regulator_driver+40>,
fn=0xffff8000107a6f60 <__driver_attach>)
at drivers/base/bus.c:305
#2 0xd6d78000107a5c58 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
I used to see more thatn 20 stacks trace but strangely I see only two.
I can still see many stacks for linux-5.4.21 that I was working with in the
past.
Could anyone check if this happens to anyone?
I think if I add BLK_DEV_RAM and set initramfs.cpio.gz in the linux build,
the kernel will boot ok to the shell prompt.
Only the gdb can't show the stack levels.
My OS : ubuntu-20.04 5.13.0-35-generic
$ aarch64-none-elf-gdb --version
GNU gdb (GNU Toolchain for the A-profile Architecture 10.2-2020.11
(arm-10.16)) 10.1.90.20201028-git
Copyright (C) 2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
<http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Thank you.
Chan Kim
_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org <mailto:Kernelnewbies@kernelnewbies.org>
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
--
regards,
Mulyadi Santosa
Freelance Linux trainer and consultant
blog: the-hydra.blogspot.com <http://the-hydra.blogspot.com>
training: mulyaditraining.blogspot.com <http://mulyaditraining.blogspot.com>
[-- Attachment #2: Type: text/html, Size: 20482 bytes --]
^ permalink raw reply [flat|nested] 12+ messages in thread
end of thread, other threads:[~2022-04-27 9:29 UTC | newest]
Thread overview: 12+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-04-22 12:14 Backtrace stopped: previous frame identical to this frame (corrupt stack?) , even with fresh qemu and linux build Chan Kim
2022-04-22 12:14 ` Chan Kim
2022-04-22 14:52 ` Mulyadi Santosa
2022-04-22 14:52 ` Mulyadi Santosa
2022-04-22 23:36 ` Chan Kim
2022-04-22 23:36 ` Chan Kim
2022-04-26 1:46 ` Chan Kim
2022-04-26 1:46 ` Chan Kim
2022-04-27 9:05 ` Chan Kim
2022-04-27 9:05 ` Chan Kim
2022-04-27 9:19 ` Chan Kim
2022-04-27 9:19 ` Chan Kim
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.