* Emulate Rpi with QEMU fails @ 2020-10-03 11:45 Thomas 2020-10-04 17:44 ` Alex Bennée 0 siblings, 1 reply; 19+ messages in thread From: Thomas @ 2020-10-03 11:45 UTC (permalink / raw) To: qemu-arm; +Cc: qemu-devel Hi, I'm trying to emulate Rpi with QEMU. I found [url=https://www.raspberry-pi-geek.de/ausgaben/rpg/2014/04/raspberry-pi-emulieren/]this[/url] arcticle in Raspberry Pi Geek documenting the steps including persistent storage on host. However when starting the emulation with command qemu-system-arm -M versatilepb -cpu arm1176 -m 256 -serial stdio -hda 2020-08-20-raspios-buster-armhf-lite.img -net "user,hostfwd=tcp::5022-:22" -dtb versatile-pb-buster.dtb -kernel kernel-qemu-5.4.51-buster -append "root=/dev/sda2 rootfstype=ext4 rw panic=1" -no-reboot I get this error: VFS: Cannot open root device "sda2" or unknown-block(0,0): error -6 Please append a correct "root=" boot option; here are the available partitions: 0100 4096 ram0 (driver?) 0101 4096 ram1 (driver?) 0102 4096 ram2 (driver?) 0103 4096 ram3 (driver?) 0104 4096 ram4 (driver?) 0105 4096 ram5 (driver?) 0106 4096 ram6 (driver?) 0107 4096 ram7 (driver?) 0108 4096 ram8 (driver?) 0109 4096 ram9 (driver?) 010a 4096 ram10 (driver?) 010b 4096 ram11 (driver?) 010c 4096 ram12 (driver?) 010d 4096 ram13 (driver?) 010e 4096 ram14 (driver?) 010f 4096 ram15 (driver?) 1f00 65536 mtdblock0 (driver?) Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0) I assume this is related to the content in fstab of RaspiOS: proc /proc proc defaults 0 0 PARTUUID=907af7d0-01 /boot vfat defaults 0 2 PARTUUID=907af7d0-02 / ext4 defaults,noatime 0 1 Can you please advise how to fix this error? THX ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Emulate Rpi with QEMU fails 2020-10-03 11:45 Emulate Rpi with QEMU fails Thomas @ 2020-10-04 17:44 ` Alex Bennée 2020-10-04 18:40 ` Peter Maydell 0 siblings, 1 reply; 19+ messages in thread From: Alex Bennée @ 2020-10-04 17:44 UTC (permalink / raw) To: Thomas Cc: Peter Maydell, qemu-arm, qemu-devel, Andrew Baumann, Philippe Mathieu-Daudé Thomas <74cmonty@gmail.com> writes: > Hi, > > I'm trying to emulate Rpi with QEMU. > I found > [url=1]this[/url] > arcticle in Raspberry Pi Geek documenting the steps including persistent > storage on host. > > However when starting the emulation with command > qemu-system-arm -M versatilepb -cpu arm1176 -m 256 -serial stdio -hda > 2020-08-20-raspios-buster-armhf-lite.img -net > "user,hostfwd=tcp::5022-:22" -dtb versatile-pb-buster.dtb -kernel > kernel-qemu-5.4.51-buster -append "root=/dev/sda2 rootfstype=ext4 rw > panic=1" -no-reboot Let's start with the fact you are using a versatilepb machine type with a versatilepb dtb and not the rasppi model. The manual goes into more details about why you can't generally share kernels built for one ARM system on another: https://qemu.readthedocs.io/en/latest/system/target-arm.html Since that article was written we have added models for the -M raspi2 and -M raspi3 which might work better with a kernel written for it although the emulation isn't complete. > I get this error: > VFS: Cannot open root device "sda2" or unknown-block(0,0): error -6 > Please append a correct "root=" boot option; here are the available > partitions: > 0100 4096 ram0 > (driver?) > 0101 4096 ram1 > (driver?) > 0102 4096 ram2 > (driver?) > 0103 4096 ram3 > (driver?) > 0104 4096 ram4 > (driver?) > 0105 4096 ram5 > (driver?) > 0106 4096 ram6 > (driver?) > 0107 4096 ram7 > (driver?) > 0108 4096 ram8 > (driver?) > 0109 4096 ram9 > (driver?) > 010a 4096 ram10 > (driver?) > 010b 4096 ram11 > (driver?) > 010c 4096 ram12 > (driver?) > 010d 4096 ram13 > (driver?) > 010e 4096 ram14 > (driver?) > 010f 4096 ram15 > (driver?) > 1f00 65536 mtdblock0 > (driver?) > Kernel panic - not syncing: VFS: Unable to mount root fs on > unknown-block(0,0) > > I assume this is related to the content in fstab of RaspiOS: > proc /proc proc defaults 0 0 > PARTUUID=907af7d0-01 /boot vfat defaults 0 2 > PARTUUID=907af7d0-02 / ext4 defaults,noatime 0 1 > > Can you please advise how to fix this error? > > THX -- Alex Bennée ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Emulate Rpi with QEMU fails 2020-10-04 17:44 ` Alex Bennée @ 2020-10-04 18:40 ` Peter Maydell 2020-10-05 9:40 ` Alex Bennée 0 siblings, 1 reply; 19+ messages in thread From: Peter Maydell @ 2020-10-04 18:40 UTC (permalink / raw) To: Alex Bennée Cc: qemu-arm, Andrew Baumann, QEMU Developers, Thomas, Philippe Mathieu-Daudé On Sun, 4 Oct 2020 at 18:44, Alex Bennée <alex.bennee@linaro.org> wrote: > Thomas <74cmonty@gmail.com> writes: > > I'm trying to emulate Rpi with QEMU. > > I found > > [url=1]this[/url] > > arcticle in Raspberry Pi Geek documenting the steps including persistent > > storage on host. > > > > However when starting the emulation with command > > qemu-system-arm -M versatilepb -cpu arm1176 -m 256 -serial stdio -hda > > 2020-08-20-raspios-buster-armhf-lite.img -net > > "user,hostfwd=tcp::5022-:22" -dtb versatile-pb-buster.dtb -kernel > > kernel-qemu-5.4.51-buster -append "root=/dev/sda2 rootfstype=ext4 rw > > panic=1" -no-reboot > > Let's start with the fact you are using a versatilepb machine type with > a versatilepb dtb and not the rasppi model. Given the name of the kernel image, this probably actually *is* built for versatilepb, or it wouldn't have got as far as failing to mount the root partition. There seems to be a lot of confusion in the raspberry pi community about the difference between running the raspi userspace plus a for-versatilepb kernel versus running a full raspi setup. Anyway, failing to mount the rootfs and not listing any sda devices is not a problem with the fstab, because the system hasn't got as far as being able to mount the filesystem with a fstab on it yet. One possibility is that the kernel is missing the device drivers for either PCI or for the SCSI controller that gets plugged in to versatilepb by default. My guess at the cause is that you're trying to boot a Linux 5.something kernel and you've run into the issue described in this thread: https://lists.gnu.org/archive/html/qemu-discuss/2020-09/msg00023.html where the Linux 5.x sym53c8xx scsi driver is not compatible with QEMU's emulation of that device. If that's the case then you should see earlier in the kernel boot log error messages similar to the ones that Roger reported. The fix would be either to use an older kernel, or to change the QEMU commandline to use a different SCSI controller (or to use a virtio block device). thanks -- PMM ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Emulate Rpi with QEMU fails 2020-10-04 18:40 ` Peter Maydell @ 2020-10-05 9:40 ` Alex Bennée 2020-10-05 10:51 ` Thomas Schneider 0 siblings, 1 reply; 19+ messages in thread From: Alex Bennée @ 2020-10-05 9:40 UTC (permalink / raw) To: Peter Maydell Cc: qemu-arm, Andrew Baumann, QEMU Developers, Thomas, Philippe Mathieu-Daudé Peter Maydell <peter.maydell@linaro.org> writes: > On Sun, 4 Oct 2020 at 18:44, Alex Bennée <alex.bennee@linaro.org> wrote: >> Thomas <74cmonty@gmail.com> writes: >> > I'm trying to emulate Rpi with QEMU. >> > I found >> > [url=1]this[/url] >> > arcticle in Raspberry Pi Geek documenting the steps including persistent >> > storage on host. >> > >> > However when starting the emulation with command >> > qemu-system-arm -M versatilepb -cpu arm1176 -m 256 -serial stdio -hda >> > 2020-08-20-raspios-buster-armhf-lite.img -net >> > "user,hostfwd=tcp::5022-:22" -dtb versatile-pb-buster.dtb -kernel >> > kernel-qemu-5.4.51-buster -append "root=/dev/sda2 rootfstype=ext4 rw >> > panic=1" -no-reboot >> >> Let's start with the fact you are using a versatilepb machine type with >> a versatilepb dtb and not the rasppi model. > > Given the name of the kernel image, this probably actually *is* > built for versatilepb, or it wouldn't have got as far as failing > to mount the root partition. There seems to be a lot of confusion > in the raspberry pi community about the difference between > running the raspi userspace plus a for-versatilepb kernel > versus running a full raspi setup. Ahh your German is considerably better than mine ;-) Looking more closely at the blog it seems to be predicated on extracting a Raspbian kernel which at least stands a fighting chance of being a multi_config kernel - like the buster above. I can see why these sorts of shenanigans used to be pulled when there where no RaspPi models although if all you want to do is run an ARM user space what's wrong with using linux-user for this sort of thing? > Anyway, failing to mount the rootfs and not listing any > sda devices is not a problem with the fstab, because the system > hasn't got as far as being able to mount the filesystem with a > fstab on it yet. One possibility is that the kernel is > missing the device drivers for either PCI or for the SCSI > controller that gets plugged in to versatilepb by default. > > My guess at the cause is that you're trying to boot a Linux 5.something > kernel and you've run into the issue described in this thread: > https://lists.gnu.org/archive/html/qemu-discuss/2020-09/msg00023.html > where the Linux 5.x sym53c8xx scsi driver is not compatible with QEMU's > emulation of that device. If that's the case then you should see > earlier in the kernel boot log error messages similar to the ones > that Roger reported. The fix would be either to use an older > kernel, or to change the QEMU commandline to use a different > SCSI controller (or to use a virtio block device). Do we have any documentation for the RaspPi models? The acceptance tests look like they support the inbuilt MMC/SD controller device: kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE + serial_kernel_cmdline[uart_id] + ' root=/dev/mmcblk0p2 rootwait ' + 'dwc_otg.fiq_fsm_enable=0') It would be useful to fill the hole in the documentation so gently steer people away from these hybrid franken-machine approaches. > > thanks > -- PMM -- Alex Bennée ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Emulate Rpi with QEMU fails 2020-10-05 9:40 ` Alex Bennée @ 2020-10-05 10:51 ` Thomas Schneider 2020-10-05 22:08 ` Paul Zimmerman 0 siblings, 1 reply; 19+ messages in thread From: Thomas Schneider @ 2020-10-05 10:51 UTC (permalink / raw) To: Alex Bennée, Peter Maydell Cc: qemu-arm, QEMU Developers, Andrew Baumann, Philippe Mathieu-Daudé Hello, thanks for your replies. I must admit that I don't fully understand your analysis. However you made some conclusions that are correct. In fact I have found a Github repo <https://github.com/dhruvvyas90/qemu-rpi-kernel> where a specific kernel and versatile-pb are provided + instructions for lauching the emulation with the original RPi image file: $ qemu-system-arm \ -M versatilepb \ -cpu arm1176 \ -m 256 \ -drive "file=/.../2020-05-27-raspios-buster-lite-armhf.img,if=none,index=0,media=disk,format=raw,id=disk0" \ -device "virtio-blk-pci,drive=disk0,disable-modern=on,disable-legacy=off" \ -net "user,hostfwd=tcp::5022-:22" \ -dtb /.../versatile-pb-buster-5.4.51.dtb \ -kernel /.../kernel-qemu-5.4.51-buster \ -append 'root=/dev/vda2 panic=1' \ -no-reboot This means it is more recent than the Raspberry Pi Geek article, and the emulation works. But I'm not sure if this usable considering the added models -M raspi2 and -M raspi3. Can you please advise how to proceed? In addition I would like to know if there's a memory limitation using models -M raspi2 and -M raspi3? To my understanding there's a limitation to 256MB using -M versatilepb. If yes, I consider to another raw image located on host's temporary filesystem and use this a swap in the client. And how can I make use of a client network device that is based on host's tap device connected to a network bridge? THX Am 05.10.2020 um 11:40 schrieb Alex Bennée: > Peter Maydell <peter.maydell@linaro.org> writes: > >> On Sun, 4 Oct 2020 at 18:44, Alex Bennée <alex.bennee@linaro.org> wrote: >>> Thomas <74cmonty@gmail.com> writes: >>>> I'm trying to emulate Rpi with QEMU. >>>> I found >>>> [url=1]this[/url] >>>> arcticle in Raspberry Pi Geek documenting the steps including persistent >>>> storage on host. >>>> >>>> However when starting the emulation with command >>>> qemu-system-arm -M versatilepb -cpu arm1176 -m 256 -serial stdio -hda >>>> 2020-08-20-raspios-buster-armhf-lite.img -net >>>> "user,hostfwd=tcp::5022-:22" -dtb versatile-pb-buster.dtb -kernel >>>> kernel-qemu-5.4.51-buster -append "root=/dev/sda2 rootfstype=ext4 rw >>>> panic=1" -no-reboot >>> Let's start with the fact you are using a versatilepb machine type with >>> a versatilepb dtb and not the rasppi model. >> Given the name of the kernel image, this probably actually *is* >> built for versatilepb, or it wouldn't have got as far as failing >> to mount the root partition. There seems to be a lot of confusion >> in the raspberry pi community about the difference between >> running the raspi userspace plus a for-versatilepb kernel >> versus running a full raspi setup. > Ahh your German is considerably better than mine ;-) Looking more > closely at the blog it seems to be predicated on extracting a Raspbian > kernel which at least stands a fighting chance of being a multi_config > kernel - like the buster above. > > I can see why these sorts of shenanigans used to be pulled when there > where no RaspPi models although if all you want to do is run an ARM user > space what's wrong with using linux-user for this sort of thing? > >> Anyway, failing to mount the rootfs and not listing any >> sda devices is not a problem with the fstab, because the system >> hasn't got as far as being able to mount the filesystem with a >> fstab on it yet. One possibility is that the kernel is >> missing the device drivers for either PCI or for the SCSI >> controller that gets plugged in to versatilepb by default. >> >> My guess at the cause is that you're trying to boot a Linux 5.something >> kernel and you've run into the issue described in this thread: >> https://lists.gnu.org/archive/html/qemu-discuss/2020-09/msg00023.html >> where the Linux 5.x sym53c8xx scsi driver is not compatible with QEMU's >> emulation of that device. If that's the case then you should see >> earlier in the kernel boot log error messages similar to the ones >> that Roger reported. The fix would be either to use an older >> kernel, or to change the QEMU commandline to use a different >> SCSI controller (or to use a virtio block device). > Do we have any documentation for the RaspPi models? The acceptance tests > look like they support the inbuilt MMC/SD controller device: > > kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE + > serial_kernel_cmdline[uart_id] + > ' root=/dev/mmcblk0p2 rootwait ' + > 'dwc_otg.fiq_fsm_enable=0') > > It would be useful to fill the hole in the documentation so gently steer > people away from these hybrid franken-machine approaches. > >> thanks >> -- PMM > ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Emulate Rpi with QEMU fails 2020-10-05 10:51 ` Thomas Schneider @ 2020-10-05 22:08 ` Paul Zimmerman 2020-10-06 6:58 ` Thomas Schneider 0 siblings, 1 reply; 19+ messages in thread From: Paul Zimmerman @ 2020-10-05 22:08 UTC (permalink / raw) To: Thomas Schneider Cc: Peter Maydell, QEMU Developers, Andrew Baumann, Philippe Mathieu-Daudé, qemu-arm, Alex Bennée If you can upgrade to Qemu 5.1 or newer, you should be able to use the builtin raspi2 or raspi3 emulation directly. Version 5.1 adds support for the USB controller on the Pi. There is no limitation on the amount of memory you can use with this method, and the networking should work. Here is the command line I use to run the Raspbian image 2019-09-26-raspbian-buster. I extracted bcm2709-rpi-2-b and kernel7.img from the FAT partition inside the image file. qemu-system-arm -M raspi2 -drive file=bootpi/2019-09-26-raspbian-buster.img,format=raw,if=sd -dtb bcm2709-rpi-2-b.dtb -kernel kernel7.img -append 'rw earlycon=pl011,0x3f201000 console=ttyAMA0 loglevel=8 root=/dev/mmcblk0p2 fsck.repair=yes net.ifnames=0 rootwait memtest=1 dwc_otg.fiq_fsm_enable=0' -serial stdio -no-reboot -netdev user,id=net0 -usb -device usb-kbd -device usb-tablet -device usb-net,netdev=net0 Hope this helps. On Mon, Oct 5, 2020 at 3:51 AM Thomas Schneider <74cmonty@gmail.com> wrote: > > Hello, > > thanks for your replies. > > I must admit that I don't fully understand your analysis. > However you made some conclusions that are correct. > > In fact I have found a Github repo > <https://github.com/dhruvvyas90/qemu-rpi-kernel> where a specific kernel > and versatile-pb are provided + instructions for lauching the emulation > with the original RPi image file: > $ qemu-system-arm \ > -M versatilepb \ > -cpu arm1176 \ > -m 256 \ > -drive > "file=/.../2020-05-27-raspios-buster-lite-armhf.img,if=none,index=0,media=disk,format=raw,id=disk0" > \ > -device > "virtio-blk-pci,drive=disk0,disable-modern=on,disable-legacy=off" \ > -net "user,hostfwd=tcp::5022-:22" \ > -dtb /.../versatile-pb-buster-5.4.51.dtb \ > -kernel /.../kernel-qemu-5.4.51-buster \ > -append 'root=/dev/vda2 panic=1' \ > -no-reboot > > This means it is more recent than the Raspberry Pi Geek article, and the > emulation works. > But I'm not sure if this usable considering the added models -M raspi2 > and -M raspi3. > > Can you please advise how to proceed? > > In addition I would like to know if there's a memory limitation using > models -M raspi2 and -M raspi3? > To my understanding there's a limitation to 256MB using -M versatilepb. > If yes, I consider to another raw image located on host's temporary > filesystem and use this a swap in the client. > > And how can I make use of a client network device that is based on > host's tap device connected to a network bridge? > > THX > > > Am 05.10.2020 um 11:40 schrieb Alex Bennée: > > Peter Maydell <peter.maydell@linaro.org> writes: > > > >> On Sun, 4 Oct 2020 at 18:44, Alex Bennée <alex.bennee@linaro.org> wrote: > >>> Thomas <74cmonty@gmail.com> writes: > >>>> I'm trying to emulate Rpi with QEMU. > >>>> I found > >>>> [url=1]this[/url] > >>>> arcticle in Raspberry Pi Geek documenting the steps including persistent > >>>> storage on host. > >>>> > >>>> However when starting the emulation with command > >>>> qemu-system-arm -M versatilepb -cpu arm1176 -m 256 -serial stdio -hda > >>>> 2020-08-20-raspios-buster-armhf-lite.img -net > >>>> "user,hostfwd=tcp::5022-:22" -dtb versatile-pb-buster.dtb -kernel > >>>> kernel-qemu-5.4.51-buster -append "root=/dev/sda2 rootfstype=ext4 rw > >>>> panic=1" -no-reboot > >>> Let's start with the fact you are using a versatilepb machine type with > >>> a versatilepb dtb and not the rasppi model. > >> Given the name of the kernel image, this probably actually *is* > >> built for versatilepb, or it wouldn't have got as far as failing > >> to mount the root partition. There seems to be a lot of confusion > >> in the raspberry pi community about the difference between > >> running the raspi userspace plus a for-versatilepb kernel > >> versus running a full raspi setup. > > Ahh your German is considerably better than mine ;-) Looking more > > closely at the blog it seems to be predicated on extracting a Raspbian > > kernel which at least stands a fighting chance of being a multi_config > > kernel - like the buster above. > > > > I can see why these sorts of shenanigans used to be pulled when there > > where no RaspPi models although if all you want to do is run an ARM user > > space what's wrong with using linux-user for this sort of thing? > > > >> Anyway, failing to mount the rootfs and not listing any > >> sda devices is not a problem with the fstab, because the system > >> hasn't got as far as being able to mount the filesystem with a > >> fstab on it yet. One possibility is that the kernel is > >> missing the device drivers for either PCI or for the SCSI > >> controller that gets plugged in to versatilepb by default. > >> > >> My guess at the cause is that you're trying to boot a Linux 5.something > >> kernel and you've run into the issue described in this thread: > >> https://lists.gnu.org/archive/html/qemu-discuss/2020-09/msg00023.html > >> where the Linux 5.x sym53c8xx scsi driver is not compatible with QEMU's > >> emulation of that device. If that's the case then you should see > >> earlier in the kernel boot log error messages similar to the ones > >> that Roger reported. The fix would be either to use an older > >> kernel, or to change the QEMU commandline to use a different > >> SCSI controller (or to use a virtio block device). > > Do we have any documentation for the RaspPi models? The acceptance tests > > look like they support the inbuilt MMC/SD controller device: > > > > kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE + > > serial_kernel_cmdline[uart_id] + > > ' root=/dev/mmcblk0p2 rootwait ' + > > 'dwc_otg.fiq_fsm_enable=0') > > > > It would be useful to fill the hole in the documentation so gently steer > > people away from these hybrid franken-machine approaches. > > > >> thanks > >> -- PMM > > > > ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Emulate Rpi with QEMU fails 2020-10-05 22:08 ` Paul Zimmerman @ 2020-10-06 6:58 ` Thomas Schneider 2020-10-06 7:42 ` Paul Zimmerman 2020-10-06 9:58 ` Alex Bennée 0 siblings, 2 replies; 19+ messages in thread From: Thomas Schneider @ 2020-10-06 6:58 UTC (permalink / raw) To: Paul Zimmerman Cc: Peter Maydell, QEMU Developers, Andrew Baumann, Philippe Mathieu-Daudé, qemu-arm, Alex Bennée Hello Paul, many thanks for sharing this info. Can you confirm that the emulated RPi with your command will use "internal QEMU" network, means the client cannot be accessed from any other device in LAN? If yes, what is required to setup a TAP connected to host's network bridge? Regards Thomas Am 06.10.2020 um 00:08 schrieb Paul Zimmerman: > If you can upgrade to Qemu 5.1 or newer, you should be able to use the > builtin raspi2 or raspi3 emulation directly. Version 5.1 adds support for the > USB controller on the Pi. > > There is no limitation on the amount of memory you can use with this > method, and the networking should work. > > Here is the command line I use to run the Raspbian image > 2019-09-26-raspbian-buster. I extracted bcm2709-rpi-2-b and > kernel7.img from the FAT partition inside the image file. > > qemu-system-arm -M raspi2 -drive > file=bootpi/2019-09-26-raspbian-buster.img,format=raw,if=sd -dtb > bcm2709-rpi-2-b.dtb -kernel kernel7.img -append 'rw > earlycon=pl011,0x3f201000 console=ttyAMA0 loglevel=8 > root=/dev/mmcblk0p2 fsck.repair=yes net.ifnames=0 rootwait memtest=1 > dwc_otg.fiq_fsm_enable=0' -serial stdio -no-reboot -netdev > user,id=net0 -usb -device usb-kbd -device usb-tablet -device > usb-net,netdev=net0 > > Hope this helps. > > On Mon, Oct 5, 2020 at 3:51 AM Thomas Schneider <74cmonty@gmail.com> wrote: >> Hello, >> >> thanks for your replies. >> >> I must admit that I don't fully understand your analysis. >> However you made some conclusions that are correct. >> >> In fact I have found a Github repo >> <https://github.com/dhruvvyas90/qemu-rpi-kernel> where a specific kernel >> and versatile-pb are provided + instructions for lauching the emulation >> with the original RPi image file: >> $ qemu-system-arm \ >> -M versatilepb \ >> -cpu arm1176 \ >> -m 256 \ >> -drive >> "file=/.../2020-05-27-raspios-buster-lite-armhf.img,if=none,index=0,media=disk,format=raw,id=disk0" >> \ >> -device >> "virtio-blk-pci,drive=disk0,disable-modern=on,disable-legacy=off" \ >> -net "user,hostfwd=tcp::5022-:22" \ >> -dtb /.../versatile-pb-buster-5.4.51.dtb \ >> -kernel /.../kernel-qemu-5.4.51-buster \ >> -append 'root=/dev/vda2 panic=1' \ >> -no-reboot >> >> This means it is more recent than the Raspberry Pi Geek article, and the >> emulation works. >> But I'm not sure if this usable considering the added models -M raspi2 >> and -M raspi3. >> >> Can you please advise how to proceed? >> >> In addition I would like to know if there's a memory limitation using >> models -M raspi2 and -M raspi3? >> To my understanding there's a limitation to 256MB using -M versatilepb. >> If yes, I consider to another raw image located on host's temporary >> filesystem and use this a swap in the client. >> >> And how can I make use of a client network device that is based on >> host's tap device connected to a network bridge? >> >> THX >> >> >> Am 05.10.2020 um 11:40 schrieb Alex Bennée: >>> Peter Maydell <peter.maydell@linaro.org> writes: >>> >>>> On Sun, 4 Oct 2020 at 18:44, Alex Bennée <alex.bennee@linaro.org> wrote: >>>>> Thomas <74cmonty@gmail.com> writes: >>>>>> I'm trying to emulate Rpi with QEMU. >>>>>> I found >>>>>> [url=1]this[/url] >>>>>> arcticle in Raspberry Pi Geek documenting the steps including persistent >>>>>> storage on host. >>>>>> >>>>>> However when starting the emulation with command >>>>>> qemu-system-arm -M versatilepb -cpu arm1176 -m 256 -serial stdio -hda >>>>>> 2020-08-20-raspios-buster-armhf-lite.img -net >>>>>> "user,hostfwd=tcp::5022-:22" -dtb versatile-pb-buster.dtb -kernel >>>>>> kernel-qemu-5.4.51-buster -append "root=/dev/sda2 rootfstype=ext4 rw >>>>>> panic=1" -no-reboot >>>>> Let's start with the fact you are using a versatilepb machine type with >>>>> a versatilepb dtb and not the rasppi model. >>>> Given the name of the kernel image, this probably actually *is* >>>> built for versatilepb, or it wouldn't have got as far as failing >>>> to mount the root partition. There seems to be a lot of confusion >>>> in the raspberry pi community about the difference between >>>> running the raspi userspace plus a for-versatilepb kernel >>>> versus running a full raspi setup. >>> Ahh your German is considerably better than mine ;-) Looking more >>> closely at the blog it seems to be predicated on extracting a Raspbian >>> kernel which at least stands a fighting chance of being a multi_config >>> kernel - like the buster above. >>> >>> I can see why these sorts of shenanigans used to be pulled when there >>> where no RaspPi models although if all you want to do is run an ARM user >>> space what's wrong with using linux-user for this sort of thing? >>> >>>> Anyway, failing to mount the rootfs and not listing any >>>> sda devices is not a problem with the fstab, because the system >>>> hasn't got as far as being able to mount the filesystem with a >>>> fstab on it yet. One possibility is that the kernel is >>>> missing the device drivers for either PCI or for the SCSI >>>> controller that gets plugged in to versatilepb by default. >>>> >>>> My guess at the cause is that you're trying to boot a Linux 5.something >>>> kernel and you've run into the issue described in this thread: >>>> https://lists.gnu.org/archive/html/qemu-discuss/2020-09/msg00023.html >>>> where the Linux 5.x sym53c8xx scsi driver is not compatible with QEMU's >>>> emulation of that device. If that's the case then you should see >>>> earlier in the kernel boot log error messages similar to the ones >>>> that Roger reported. The fix would be either to use an older >>>> kernel, or to change the QEMU commandline to use a different >>>> SCSI controller (or to use a virtio block device). >>> Do we have any documentation for the RaspPi models? The acceptance tests >>> look like they support the inbuilt MMC/SD controller device: >>> >>> kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE + >>> serial_kernel_cmdline[uart_id] + >>> ' root=/dev/mmcblk0p2 rootwait ' + >>> 'dwc_otg.fiq_fsm_enable=0') >>> >>> It would be useful to fill the hole in the documentation so gently steer >>> people away from these hybrid franken-machine approaches. >>> >>>> thanks >>>> -- PMM >> ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Emulate Rpi with QEMU fails 2020-10-06 6:58 ` Thomas Schneider @ 2020-10-06 7:42 ` Paul Zimmerman 2020-10-06 9:58 ` Alex Bennée 1 sibling, 0 replies; 19+ messages in thread From: Paul Zimmerman @ 2020-10-06 7:42 UTC (permalink / raw) To: Thomas Schneider Cc: Peter Maydell, QEMU Developers, Andrew Baumann, Philippe Mathieu-Daudé, qemu-arm, Alex Bennée On Mon, Oct 5, 2020 at 11:58 PM Thomas Schneider <74cmonty@gmail.com> wrote: > > Hello Paul, > > many thanks for sharing this info. > > Can you confirm that the emulated RPi with your command will use > "internal QEMU" network, means the client cannot be accessed from any > other device in LAN? > If yes, what is required to setup a TAP connected to host's network bridge? > > Regards > Thomas That sets up User networking, so yes, generally you cannot access the client from the outside network. If you set up a bridge device on the host using TAP, then you can change the "-netdev user,id=net0" in the command line to "-netdev tap,helper=/usr/local/libexec/qemu-bridge-helper,id=net0" (assuming your qemu was installed in /usr/local) and you should get a fully functional network connection. I found that the bridge device must be named "br0" or this will not work. I found that setting up a bridge device is pretty finicky, though. You can google for instructions on how to do it. I *think* these are the steps I followed to make it work for me. This is on Ubuntu, other Linux distributions work differently I believe. My ethernet device is 'eth0' and my host ip address is '192.168.0.54', so you should change those as needed for your environment. sudo ip link add br0 type bridge sudo ip tuntap add dev tap0 mode tap sudo ip link set dev tap0 master br0 sudo ip link set dev eth0 master br0 sudo ip link set dev br0 up sudo ip address delete 192.168.0.54/24 dev eth0 sudo ip address add 192.168.0.54/24 dev br0 sudo ip route add default via 192.168.0.1 dev br0 - Paul > > Am 06.10.2020 um 00:08 schrieb Paul Zimmerman: > > If you can upgrade to Qemu 5.1 or newer, you should be able to use the > > builtin raspi2 or raspi3 emulation directly. Version 5.1 adds support for the > > USB controller on the Pi. > > > > There is no limitation on the amount of memory you can use with this > > method, and the networking should work. > > > > Here is the command line I use to run the Raspbian image > > 2019-09-26-raspbian-buster. I extracted bcm2709-rpi-2-b and > > kernel7.img from the FAT partition inside the image file. > > > > qemu-system-arm -M raspi2 -drive > > file=bootpi/2019-09-26-raspbian-buster.img,format=raw,if=sd -dtb > > bcm2709-rpi-2-b.dtb -kernel kernel7.img -append 'rw > > earlycon=pl011,0x3f201000 console=ttyAMA0 loglevel=8 > > root=/dev/mmcblk0p2 fsck.repair=yes net.ifnames=0 rootwait memtest=1 > > dwc_otg.fiq_fsm_enable=0' -serial stdio -no-reboot -netdev > > user,id=net0 -usb -device usb-kbd -device usb-tablet -device > > usb-net,netdev=net0 > > > > Hope this helps. > > > > On Mon, Oct 5, 2020 at 3:51 AM Thomas Schneider <74cmonty@gmail.com> wrote: > >> Hello, > >> > >> thanks for your replies. > >> > >> I must admit that I don't fully understand your analysis. > >> However you made some conclusions that are correct. > >> > >> In fact I have found a Github repo > >> <https://github.com/dhruvvyas90/qemu-rpi-kernel> where a specific kernel > >> and versatile-pb are provided + instructions for lauching the emulation > >> with the original RPi image file: > >> $ qemu-system-arm \ > >> -M versatilepb \ > >> -cpu arm1176 \ > >> -m 256 \ > >> -drive > >> "file=/.../2020-05-27-raspios-buster-lite-armhf.img,if=none,index=0,media=disk,format=raw,id=disk0" > >> \ > >> -device > >> "virtio-blk-pci,drive=disk0,disable-modern=on,disable-legacy=off" \ > >> -net "user,hostfwd=tcp::5022-:22" \ > >> -dtb /.../versatile-pb-buster-5.4.51.dtb \ > >> -kernel /.../kernel-qemu-5.4.51-buster \ > >> -append 'root=/dev/vda2 panic=1' \ > >> -no-reboot > >> > >> This means it is more recent than the Raspberry Pi Geek article, and the > >> emulation works. > >> But I'm not sure if this usable considering the added models -M raspi2 > >> and -M raspi3. > >> > >> Can you please advise how to proceed? > >> > >> In addition I would like to know if there's a memory limitation using > >> models -M raspi2 and -M raspi3? > >> To my understanding there's a limitation to 256MB using -M versatilepb. > >> If yes, I consider to another raw image located on host's temporary > >> filesystem and use this a swap in the client. > >> > >> And how can I make use of a client network device that is based on > >> host's tap device connected to a network bridge? > >> > >> THX > >> > >> > >> Am 05.10.2020 um 11:40 schrieb Alex Bennée: > >>> Peter Maydell <peter.maydell@linaro.org> writes: > >>> > >>>> On Sun, 4 Oct 2020 at 18:44, Alex Bennée <alex.bennee@linaro.org> wrote: > >>>>> Thomas <74cmonty@gmail.com> writes: > >>>>>> I'm trying to emulate Rpi with QEMU. > >>>>>> I found > >>>>>> [url=1]this[/url] > >>>>>> arcticle in Raspberry Pi Geek documenting the steps including persistent > >>>>>> storage on host. > >>>>>> > >>>>>> However when starting the emulation with command > >>>>>> qemu-system-arm -M versatilepb -cpu arm1176 -m 256 -serial stdio -hda > >>>>>> 2020-08-20-raspios-buster-armhf-lite.img -net > >>>>>> "user,hostfwd=tcp::5022-:22" -dtb versatile-pb-buster.dtb -kernel > >>>>>> kernel-qemu-5.4.51-buster -append "root=/dev/sda2 rootfstype=ext4 rw > >>>>>> panic=1" -no-reboot > >>>>> Let's start with the fact you are using a versatilepb machine type with > >>>>> a versatilepb dtb and not the rasppi model. > >>>> Given the name of the kernel image, this probably actually *is* > >>>> built for versatilepb, or it wouldn't have got as far as failing > >>>> to mount the root partition. There seems to be a lot of confusion > >>>> in the raspberry pi community about the difference between > >>>> running the raspi userspace plus a for-versatilepb kernel > >>>> versus running a full raspi setup. > >>> Ahh your German is considerably better than mine ;-) Looking more > >>> closely at the blog it seems to be predicated on extracting a Raspbian > >>> kernel which at least stands a fighting chance of being a multi_config > >>> kernel - like the buster above. > >>> > >>> I can see why these sorts of shenanigans used to be pulled when there > >>> where no RaspPi models although if all you want to do is run an ARM user > >>> space what's wrong with using linux-user for this sort of thing? > >>> > >>>> Anyway, failing to mount the rootfs and not listing any > >>>> sda devices is not a problem with the fstab, because the system > >>>> hasn't got as far as being able to mount the filesystem with a > >>>> fstab on it yet. One possibility is that the kernel is > >>>> missing the device drivers for either PCI or for the SCSI > >>>> controller that gets plugged in to versatilepb by default. > >>>> > >>>> My guess at the cause is that you're trying to boot a Linux 5.something > >>>> kernel and you've run into the issue described in this thread: > >>>> https://lists.gnu.org/archive/html/qemu-discuss/2020-09/msg00023.html > >>>> where the Linux 5.x sym53c8xx scsi driver is not compatible with QEMU's > >>>> emulation of that device. If that's the case then you should see > >>>> earlier in the kernel boot log error messages similar to the ones > >>>> that Roger reported. The fix would be either to use an older > >>>> kernel, or to change the QEMU commandline to use a different > >>>> SCSI controller (or to use a virtio block device). > >>> Do we have any documentation for the RaspPi models? The acceptance tests > >>> look like they support the inbuilt MMC/SD controller device: > >>> > >>> kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE + > >>> serial_kernel_cmdline[uart_id] + > >>> ' root=/dev/mmcblk0p2 rootwait ' + > >>> 'dwc_otg.fiq_fsm_enable=0') > >>> > >>> It would be useful to fill the hole in the documentation so gently steer > >>> people away from these hybrid franken-machine approaches. > >>> > >>>> thanks > >>>> -- PMM > >> > ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Emulate Rpi with QEMU fails 2020-10-06 6:58 ` Thomas Schneider 2020-10-06 7:42 ` Paul Zimmerman @ 2020-10-06 9:58 ` Alex Bennée 2020-10-07 6:28 ` Thomas 1 sibling, 1 reply; 19+ messages in thread From: Alex Bennée @ 2020-10-06 9:58 UTC (permalink / raw) To: Thomas Schneider Cc: Peter Maydell, Philippe Mathieu-Daudé, Andrew Baumann, QEMU Developers, qemu-arm, Paul Zimmerman Thomas Schneider <74cmonty@gmail.com> writes: > Hello Paul, > > many thanks for sharing this info. > > Can you confirm that the emulated RPi with your command will use > "internal QEMU" network, means the client cannot be accessed from any > other device in LAN? The support for user-mode and TAP networking is orthogonal to the emulated device. However if you only want a few ports it's quite easy to use port forwarding, e.g: -netdev user,id=unet,hostfwd=tcp::2222-:22 which forwards 2222 to port 22 on the device. I have an alias in .ssh/config for accessing my QEMU devices. > If yes, what is required to setup a TAP connected to host's network > bridge? I'll defer to others for this but generally when I want proper bridged networking for a VM I use virt-manager/libvirt to configure it because it can be quite fiddly to do by hand. -- Alex Bennée ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Emulate Rpi with QEMU fails 2020-10-06 9:58 ` Alex Bennée @ 2020-10-07 6:28 ` Thomas 2020-10-07 6:50 ` Paul Zimmerman 0 siblings, 1 reply; 19+ messages in thread From: Thomas @ 2020-10-07 6:28 UTC (permalink / raw) To: Alex Bennée Cc: Peter Maydell, Philippe Mathieu-Daudé, Andrew Baumann, QEMU Developers, qemu-arm, Paul Zimmerman Hello! Many thanks for your support. I managed to get emulated RPi starting. However there's one question I want to ask: How can I accelerate the startup sequence? I mean booting the emulated RPi takes more than 3 minutes. Regards Thomas Am 06.10.20 um 11:58 schrieb Alex Bennée: > Thomas Schneider <74cmonty@gmail.com> writes: > >> Hello Paul, >> >> many thanks for sharing this info. >> >> Can you confirm that the emulated RPi with your command will use >> "internal QEMU" network, means the client cannot be accessed from any >> other device in LAN? > The support for user-mode and TAP networking is orthogonal to the > emulated device. However if you only want a few ports it's quite easy to > use port forwarding, e.g: > > -netdev user,id=unet,hostfwd=tcp::2222-:22 > > which forwards 2222 to port 22 on the device. I have an alias in > .ssh/config for accessing my QEMU devices. > >> If yes, what is required to setup a TAP connected to host's network >> bridge? > I'll defer to others for this but generally when I want proper bridged > networking for a VM I use virt-manager/libvirt to configure it because > it can be quite fiddly to do by hand. > ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Emulate Rpi with QEMU fails 2020-10-07 6:28 ` Thomas @ 2020-10-07 6:50 ` Paul Zimmerman 2020-10-07 7:27 ` Thomas Schneider 0 siblings, 1 reply; 19+ messages in thread From: Paul Zimmerman @ 2020-10-07 6:50 UTC (permalink / raw) To: Thomas Cc: Peter Maydell, QEMU Developers, Andrew Baumann, Philippe Mathieu-Daudé, qemu-arm, Alex Bennée On Tue, Oct 6, 2020 at 11:28 PM Thomas <74cmonty@gmail.com> wrote: > > Hello! > > Many thanks for your support. > > I managed to get emulated RPi starting. > > However there's one question I want to ask: > How can I accelerate the startup sequence? > I mean booting the emulated RPi takes more than 3 minutes. > > Regards > Thomas Get a faster computer? ;) On my Intel i7 desktop it takes about 40 seconds to boot to the login: prompt on the serial console, and about 1 min 8 seconds before the GUI is up. On my 5 year old laptop it's probably twice that. I don't know of any way to make it go faster. - Paul > > Am 06.10.20 um 11:58 schrieb Alex Bennée: > > Thomas Schneider <74cmonty@gmail.com> writes: > > > >> Hello Paul, > >> > >> many thanks for sharing this info. > >> > >> Can you confirm that the emulated RPi with your command will use > >> "internal QEMU" network, means the client cannot be accessed from any > >> other device in LAN? > > The support for user-mode and TAP networking is orthogonal to the > > emulated device. However if you only want a few ports it's quite easy to > > use port forwarding, e.g: > > > > -netdev user,id=unet,hostfwd=tcp::2222-:22 > > > > which forwards 2222 to port 22 on the device. I have an alias in > > .ssh/config for accessing my QEMU devices. > > > >> If yes, what is required to setup a TAP connected to host's network > >> bridge? > > I'll defer to others for this but generally when I want proper bridged > > networking for a VM I use virt-manager/libvirt to configure it because > > it can be quite fiddly to do by hand. > > > ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Emulate Rpi with QEMU fails 2020-10-07 6:50 ` Paul Zimmerman @ 2020-10-07 7:27 ` Thomas Schneider 2020-10-07 11:00 ` Alex Bennée 0 siblings, 1 reply; 19+ messages in thread From: Thomas Schneider @ 2020-10-07 7:27 UTC (permalink / raw) To: Paul Zimmerman Cc: Peter Maydell, QEMU Developers, Andrew Baumann, Philippe Mathieu-Daudé, qemu-arm, Alex Bennée Hi, I already considered the host CPU power. However I have this <https://ark.intel.com/content/www/us/en/ark/products/33924/intel-core-2-quad-processor-q9550-12m-cache-2-83-ghz-1333-mhz-fsb.html> CPU Intel Core 2 Quad Q9550 2,83 GHz and assumed this should be powerful enough for RPi emulation. But maybe my assumption was too optimistic. Am 07.10.2020 um 08:50 schrieb Paul Zimmerman: > On Tue, Oct 6, 2020 at 11:28 PM Thomas <74cmonty@gmail.com> wrote: >> Hello! >> >> Many thanks for your support. >> >> I managed to get emulated RPi starting. >> >> However there's one question I want to ask: >> How can I accelerate the startup sequence? >> I mean booting the emulated RPi takes more than 3 minutes. >> >> Regards >> Thomas > Get a faster computer? ;) > > On my Intel i7 desktop it takes about 40 seconds to boot to the login: > prompt on the serial console, and about 1 min 8 seconds before the > GUI is up. On my 5 year old laptop it's probably twice that. I don't know > of any way to make it go faster. > > - Paul > >> Am 06.10.20 um 11:58 schrieb Alex Bennée: >>> Thomas Schneider <74cmonty@gmail.com> writes: >>> >>>> Hello Paul, >>>> >>>> many thanks for sharing this info. >>>> >>>> Can you confirm that the emulated RPi with your command will use >>>> "internal QEMU" network, means the client cannot be accessed from any >>>> other device in LAN? >>> The support for user-mode and TAP networking is orthogonal to the >>> emulated device. However if you only want a few ports it's quite easy to >>> use port forwarding, e.g: >>> >>> -netdev user,id=unet,hostfwd=tcp::2222-:22 >>> >>> which forwards 2222 to port 22 on the device. I have an alias in >>> .ssh/config for accessing my QEMU devices. >>> >>>> If yes, what is required to setup a TAP connected to host's network >>>> bridge? >>> I'll defer to others for this but generally when I want proper bridged >>> networking for a VM I use virt-manager/libvirt to configure it because >>> it can be quite fiddly to do by hand. >>> ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Emulate Rpi with QEMU fails 2020-10-07 7:27 ` Thomas Schneider @ 2020-10-07 11:00 ` Alex Bennée 2020-10-07 11:36 ` Thomas Schneider 0 siblings, 1 reply; 19+ messages in thread From: Alex Bennée @ 2020-10-07 11:00 UTC (permalink / raw) To: Thomas Schneider Cc: Peter Maydell, Philippe Mathieu-Daudé, Andrew Baumann, QEMU Developers, qemu-arm, Paul Zimmerman Thomas Schneider <74cmonty@gmail.com> writes: > Hi, > > I already considered the host CPU power. > However I have this > <https://ark.intel.com/content/www/us/en/ark/products/33924/intel-core-2-quad-processor-q9550-12m-cache-2-83-ghz-1333-mhz-fsb.html> > CPU > Intel Core 2 Quad Q9550 2,83 GHz > and assumed this should be powerful enough for RPi emulation. For each emulated instruction you can be running between 6-10 host instructions on average. We have certainly improved the performance of the emulation over time and take advantage of multiple threads but in the end system emulation will always be fairly expensive. > But maybe my assumption was too optimistic. You can use perf to record your boot and analyse where QEMU is spending it's time. Unless there is a major outlier though it's unlikely to be easy to optimise. > > > Am 07.10.2020 um 08:50 schrieb Paul Zimmerman: >> On Tue, Oct 6, 2020 at 11:28 PM Thomas <74cmonty@gmail.com> wrote: >>> Hello! >>> >>> Many thanks for your support. >>> >>> I managed to get emulated RPi starting. >>> >>> However there's one question I want to ask: >>> How can I accelerate the startup sequence? >>> I mean booting the emulated RPi takes more than 3 minutes. >>> >>> Regards >>> Thomas >> Get a faster computer? ;) >> >> On my Intel i7 desktop it takes about 40 seconds to boot to the login: >> prompt on the serial console, and about 1 min 8 seconds before the >> GUI is up. On my 5 year old laptop it's probably twice that. I don't know >> of any way to make it go faster. >> >> - Paul >> >>> Am 06.10.20 um 11:58 schrieb Alex Bennée: >>>> Thomas Schneider <74cmonty@gmail.com> writes: >>>> >>>>> Hello Paul, >>>>> >>>>> many thanks for sharing this info. >>>>> >>>>> Can you confirm that the emulated RPi with your command will use >>>>> "internal QEMU" network, means the client cannot be accessed from any >>>>> other device in LAN? >>>> The support for user-mode and TAP networking is orthogonal to the >>>> emulated device. However if you only want a few ports it's quite easy to >>>> use port forwarding, e.g: >>>> >>>> -netdev user,id=unet,hostfwd=tcp::2222-:22 >>>> >>>> which forwards 2222 to port 22 on the device. I have an alias in >>>> .ssh/config for accessing my QEMU devices. >>>> >>>>> If yes, what is required to setup a TAP connected to host's network >>>>> bridge? >>>> I'll defer to others for this but generally when I want proper bridged >>>> networking for a VM I use virt-manager/libvirt to configure it because >>>> it can be quite fiddly to do by hand. >>>> -- Alex Bennée ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Emulate Rpi with QEMU fails 2020-10-07 11:00 ` Alex Bennée @ 2020-10-07 11:36 ` Thomas Schneider 2020-10-07 12:02 ` Alex Bennée 0 siblings, 1 reply; 19+ messages in thread From: Thomas Schneider @ 2020-10-07 11:36 UTC (permalink / raw) To: Alex Bennée Cc: Peter Maydell, Philippe Mathieu-Daudé, Andrew Baumann, QEMU Developers, qemu-arm, Paul Zimmerman Are you referring to this tool? https://github.com/stefano-garzarella/qemu-boot-time <https://github.com/stefano-garzarella/qemu-boot-time> Am 07.10.2020 um 13:00 schrieb Alex Bennée: > perf to record your boot ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Emulate Rpi with QEMU fails 2020-10-07 11:36 ` Thomas Schneider @ 2020-10-07 12:02 ` Alex Bennée 2020-10-08 7:00 ` Thomas 0 siblings, 1 reply; 19+ messages in thread From: Alex Bennée @ 2020-10-07 12:02 UTC (permalink / raw) To: Thomas Schneider Cc: Peter Maydell, Philippe Mathieu-Daudé, Andrew Baumann, QEMU Developers, qemu-arm, Paul Zimmerman Thomas Schneider <74cmonty@gmail.com> writes: > Are you referring to this tool? > https://github.com/stefano-garzarella/qemu-boot-time > <https://github.com/stefano-garzarella/qemu-boot-time> No - just plain perf: perf record $QEMU $ARGS Then a "perf report" which will show you the hotspots, for example: 8.92% qemu-system-aar qemu-system-aarch64 [.] helper_lookup_tb_ptr 4.76% qemu-system-aar qemu-system-aarch64 [.] liveness_pass_1 3.69% qemu-system-aar qemu-system-aarch64 [.] tcg_gen_code 2.95% qemu-system-aar qemu-system-aarch64 [.] qht_lookup_custom 2.93% qemu-system-aar qemu-system-aarch64 [.] tcg_optimize 1.28% qemu-system-aar qemu-system-aarch64 [.] tcg_out_opc.isra.15 1.09% qemu-system-aar qemu-system-aarch64 [.] get_phys_addr_lpae 1.09% qemu-system-aar [kernel.kallsyms] [k] isolate_freepages_block 1.05% qemu-system-aar qemu-system-aarch64 [.] cpu_get_tb_cpu_state 0.98% qemu-system-aar [kernel.kallsyms] [k] do_syscall_64 0.94% qemu-system-aar qemu-system-aarch64 [.] tb_lookup_cmp 0.78% qemu-system-aar qemu-system-aarch64 [.] init_ts_info 0.73% qemu-system-aar qemu-system-aarch64 [.] tb_htable_lookup 0.73% qemu-system-aar qemu-system-aarch64 [.] tb_gen_code 0.73% qemu-system-aar qemu-system-aarch64 [.] tlb_set_page_with_attrs > > > Am 07.10.2020 um 13:00 schrieb Alex Bennée: >> perf to record your boot -- Alex Bennée ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Emulate Rpi with QEMU fails 2020-10-07 12:02 ` Alex Bennée @ 2020-10-08 7:00 ` Thomas 2020-10-08 21:07 ` Paul Zimmerman 2020-10-09 6:20 ` Alex Bennée 0 siblings, 2 replies; 19+ messages in thread From: Thomas @ 2020-10-08 7:00 UTC (permalink / raw) To: Alex Bennée Cc: Peter Maydell, Philippe Mathieu-Daudé, Andrew Baumann, QEMU Developers, qemu-arm, Paul Zimmerman Interesting enough is: my top figure reported by perf is like yours: Samples: 6M of event 'cycles:u', Event count (approx.): 1936571734942 Overhead Command Shared Object Symbol 7,95% qemu-system-arm qemu-system-arm [.] helper_lookup_tb_ptr ◆ 4,16% qemu-system-arm qemu-system-arm [.] cpu_get_tb_cpu_state ▒ 2,52% qemu-system-arm libpthread-2.32.so [.] __pthread_mutex_lock ▒ 2,06% qemu-system-arm qemu-system-arm [.] qht_lookup_custom ▒ 1,66% qemu-system-arm qemu-system-arm [.] tlb_set_page_with_attrs ▒ 1,61% qemu-system-arm libpthread-2.32.so [.] __pthread_mutex_unlock_usercnt ▒ 1,59% qemu-system-arm qemu-system-arm [.] get_phys_addr ▒ 1,27% qemu-system-arm qemu-system-arm [.] cpu_exec ▒ 1,23% qemu-system-arm qemu-system-arm [.] object_class_dynamic_cast_assert ▒ 0,98% qemu-system-arm libc-2.32.so [.] _int_malloc ▒ 0,95% qemu-system-arm qemu-system-arm [.] object_dynamic_cast_assert ▒ 0,95% qemu-system-arm qemu-system-arm [.] tb_htable_lookup ▒ 0,92% qemu-system-arm qemu-system-arm [.] tcg_gen_code ▒ 0,91% qemu-system-arm qemu-system-arm [.] qemu_mutex_lock_impl ▒ 0,75% qemu-system-arm qemu-system-arm [.] get_page_addr_code_hostp ▒ 0,73% qemu-system-arm qemu-system-arm [.] tcg_optimize ▒ 0,71% qemu-system-arm qemu-system-arm [.] qemu_mutex_unlock_impl ▒ 0,69% qemu-system-arm libc-2.32.so [.] _int_free ▒ 0,64% qemu-system-arm qemu-system-arm [.] tb_flush_jmp_cache ▒ 0,63% qemu-system-arm qemu-system-arm [.] address_space_ldl_le ▒ 0,62% qemu-system-arm libpthread-2.32.so [.] __lll_lock_wait ▒ 0,58% qemu-system-arm libpthread-2.32.so [.] pthread_cond_wait@@GLIBC_2.3.2 ▒ 0,52% qemu-system-arm qemu-system-arm [.] cpu_reset_interrupt ▒ 0,52% qemu-system-arm libc-2.32.so [.] cfree@GLIBC_2.2.5 ▒ 0,52% qemu-system-arm qemu-system-arm [.] qemu_set_irq ▒ However the absolute time is 3-4 minutes. And I don't know where I should start optimization now. Am 07.10.20 um 14:02 schrieb Alex Bennée: > Thomas Schneider <74cmonty@gmail.com> writes: > >> Are you referring to this tool? >> https://github.com/stefano-garzarella/qemu-boot-time >> <https://github.com/stefano-garzarella/qemu-boot-time> > No - just plain perf: > > perf record $QEMU $ARGS > > Then a "perf report" which will show you the hotspots, for example: > > 8.92% qemu-system-aar qemu-system-aarch64 [.] helper_lookup_tb_ptr > 4.76% qemu-system-aar qemu-system-aarch64 [.] liveness_pass_1 > 3.69% qemu-system-aar qemu-system-aarch64 [.] tcg_gen_code > 2.95% qemu-system-aar qemu-system-aarch64 [.] qht_lookup_custom > 2.93% qemu-system-aar qemu-system-aarch64 [.] tcg_optimize > 1.28% qemu-system-aar qemu-system-aarch64 [.] tcg_out_opc.isra.15 > 1.09% qemu-system-aar qemu-system-aarch64 [.] get_phys_addr_lpae > 1.09% qemu-system-aar [kernel.kallsyms] [k] isolate_freepages_block > 1.05% qemu-system-aar qemu-system-aarch64 [.] cpu_get_tb_cpu_state > 0.98% qemu-system-aar [kernel.kallsyms] [k] do_syscall_64 > 0.94% qemu-system-aar qemu-system-aarch64 [.] tb_lookup_cmp > 0.78% qemu-system-aar qemu-system-aarch64 [.] init_ts_info > 0.73% qemu-system-aar qemu-system-aarch64 [.] tb_htable_lookup > 0.73% qemu-system-aar qemu-system-aarch64 [.] tb_gen_code > 0.73% qemu-system-aar qemu-system-aarch64 [.] tlb_set_page_with_attrs > >> >> Am 07.10.2020 um 13:00 schrieb Alex Bennée: >>> perf to record your boot > ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Emulate Rpi with QEMU fails 2020-10-08 7:00 ` Thomas @ 2020-10-08 21:07 ` Paul Zimmerman 2020-10-09 2:21 ` Paul Zimmerman 2020-10-09 6:20 ` Alex Bennée 1 sibling, 1 reply; 19+ messages in thread From: Paul Zimmerman @ 2020-10-08 21:07 UTC (permalink / raw) To: Thomas Cc: Peter Maydell, QEMU Developers, Andrew Baumann, Philippe Mathieu-Daudé, qemu-arm, Alex Bennée Hi Thomas, What does 'top' say while the emulation is running? I have an 8 cpu-thread system, yet 'top' never shows more than about 300% cpu. I would have thought it would get closer to 800% cpu. And running qemu as root with nice -20 doesn't seem to make much difference. - Paul On Thu, Oct 8, 2020 at 12:00 AM Thomas <74cmonty@gmail.com> wrote: > > Interesting enough is: my top figure reported by perf is like yours: > > Samples: 6M of event 'cycles:u', Event count (approx.): 1936571734942 > Overhead Command Shared Object Symbol > 7,95% qemu-system-arm qemu-system-arm [.] > helper_lookup_tb_ptr > ◆ > 4,16% qemu-system-arm qemu-system-arm [.] > cpu_get_tb_cpu_state > ▒ > 2,52% qemu-system-arm libpthread-2.32.so [.] > __pthread_mutex_lock > ▒ > 2,06% qemu-system-arm qemu-system-arm [.] > qht_lookup_custom > ▒ > 1,66% qemu-system-arm qemu-system-arm [.] > tlb_set_page_with_attrs > ▒ > 1,61% qemu-system-arm libpthread-2.32.so [.] > __pthread_mutex_unlock_usercnt > ▒ > 1,59% qemu-system-arm qemu-system-arm [.] > get_phys_addr > ▒ > 1,27% qemu-system-arm qemu-system-arm [.] > cpu_exec > ▒ > 1,23% qemu-system-arm qemu-system-arm [.] > object_class_dynamic_cast_assert > ▒ > 0,98% qemu-system-arm libc-2.32.so [.] > _int_malloc > ▒ > 0,95% qemu-system-arm qemu-system-arm [.] > object_dynamic_cast_assert > ▒ > 0,95% qemu-system-arm qemu-system-arm [.] > tb_htable_lookup > ▒ > 0,92% qemu-system-arm qemu-system-arm [.] > tcg_gen_code > ▒ > 0,91% qemu-system-arm qemu-system-arm [.] > qemu_mutex_lock_impl > ▒ > 0,75% qemu-system-arm qemu-system-arm [.] > get_page_addr_code_hostp > ▒ > 0,73% qemu-system-arm qemu-system-arm [.] > tcg_optimize > ▒ > 0,71% qemu-system-arm qemu-system-arm [.] > qemu_mutex_unlock_impl > ▒ > 0,69% qemu-system-arm libc-2.32.so [.] > _int_free > ▒ > 0,64% qemu-system-arm qemu-system-arm [.] > tb_flush_jmp_cache > ▒ > 0,63% qemu-system-arm qemu-system-arm [.] > address_space_ldl_le > ▒ > 0,62% qemu-system-arm libpthread-2.32.so [.] > __lll_lock_wait > ▒ > 0,58% qemu-system-arm libpthread-2.32.so [.] > pthread_cond_wait@@GLIBC_2.3.2 > ▒ > 0,52% qemu-system-arm qemu-system-arm [.] > cpu_reset_interrupt > ▒ > 0,52% qemu-system-arm libc-2.32.so [.] > cfree@GLIBC_2.2.5 > ▒ > 0,52% qemu-system-arm qemu-system-arm [.] > qemu_set_irq > ▒ > > However the absolute time is 3-4 minutes. > And I don't know where I should start optimization now. > > > > Am 07.10.20 um 14:02 schrieb Alex Bennée: > > Thomas Schneider <74cmonty@gmail.com> writes: > > > >> Are you referring to this tool? > >> https://github.com/stefano-garzarella/qemu-boot-time > >> <https://github.com/stefano-garzarella/qemu-boot-time> > > No - just plain perf: > > > > perf record $QEMU $ARGS > > > > Then a "perf report" which will show you the hotspots, for example: > > > > 8.92% qemu-system-aar qemu-system-aarch64 [.] helper_lookup_tb_ptr > > 4.76% qemu-system-aar qemu-system-aarch64 [.] liveness_pass_1 > > 3.69% qemu-system-aar qemu-system-aarch64 [.] tcg_gen_code > > 2.95% qemu-system-aar qemu-system-aarch64 [.] qht_lookup_custom > > 2.93% qemu-system-aar qemu-system-aarch64 [.] tcg_optimize > > 1.28% qemu-system-aar qemu-system-aarch64 [.] tcg_out_opc.isra.15 > > 1.09% qemu-system-aar qemu-system-aarch64 [.] get_phys_addr_lpae > > 1.09% qemu-system-aar [kernel.kallsyms] [k] isolate_freepages_block > > 1.05% qemu-system-aar qemu-system-aarch64 [.] cpu_get_tb_cpu_state > > 0.98% qemu-system-aar [kernel.kallsyms] [k] do_syscall_64 > > 0.94% qemu-system-aar qemu-system-aarch64 [.] tb_lookup_cmp > > 0.78% qemu-system-aar qemu-system-aarch64 [.] init_ts_info > > 0.73% qemu-system-aar qemu-system-aarch64 [.] tb_htable_lookup > > 0.73% qemu-system-aar qemu-system-aarch64 [.] tb_gen_code > > 0.73% qemu-system-aar qemu-system-aarch64 [.] tlb_set_page_with_attrs > > > >> > >> Am 07.10.2020 um 13:00 schrieb Alex Bennée: > >>> perf to record your boot > > > ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Emulate Rpi with QEMU fails 2020-10-08 21:07 ` Paul Zimmerman @ 2020-10-09 2:21 ` Paul Zimmerman 0 siblings, 0 replies; 19+ messages in thread From: Paul Zimmerman @ 2020-10-09 2:21 UTC (permalink / raw) To: Thomas Cc: Peter Maydell, QEMU Developers, Andrew Baumann, Philippe Mathieu-Daudé, qemu-arm, Alex Bennée Running 'top -H' (to show individual threads) I see qemu is using 6 to 7 threads while running, and each thread is taking from 15% to 70% or so of cpu time. So probably qemu is not able to spread the work among the threads evenly enough to use all the cpu time available. Might be an interesting area of investigation for someone motivated enough :) - Paul On Thu, Oct 8, 2020 at 2:07 PM Paul Zimmerman <pauldzim@gmail.com> wrote: > > Hi Thomas, > > What does 'top' say while the emulation is running? I have an 8 cpu-thread > system, yet 'top' never shows more than about 300% cpu. I would have > thought it would get closer to 800% cpu. And running qemu as root with > nice -20 doesn't seem to make much difference. > > - Paul > > On Thu, Oct 8, 2020 at 12:00 AM Thomas <74cmonty@gmail.com> wrote: > > > > Interesting enough is: my top figure reported by perf is like yours: > > > > Samples: 6M of event 'cycles:u', Event count (approx.): 1936571734942 > > Overhead Command Shared Object Symbol > > 7,95% qemu-system-arm qemu-system-arm [.] > > helper_lookup_tb_ptr > > ◆ > > 4,16% qemu-system-arm qemu-system-arm [.] > > cpu_get_tb_cpu_state > > ▒ > > 2,52% qemu-system-arm libpthread-2.32.so [.] > > __pthread_mutex_lock > > ▒ > > 2,06% qemu-system-arm qemu-system-arm [.] > > qht_lookup_custom > > ▒ > > 1,66% qemu-system-arm qemu-system-arm [.] > > tlb_set_page_with_attrs > > ▒ > > 1,61% qemu-system-arm libpthread-2.32.so [.] > > __pthread_mutex_unlock_usercnt > > ▒ > > 1,59% qemu-system-arm qemu-system-arm [.] > > get_phys_addr > > ▒ > > 1,27% qemu-system-arm qemu-system-arm [.] > > cpu_exec > > ▒ > > 1,23% qemu-system-arm qemu-system-arm [.] > > object_class_dynamic_cast_assert > > ▒ > > 0,98% qemu-system-arm libc-2.32.so [.] > > _int_malloc > > ▒ > > 0,95% qemu-system-arm qemu-system-arm [.] > > object_dynamic_cast_assert > > ▒ > > 0,95% qemu-system-arm qemu-system-arm [.] > > tb_htable_lookup > > ▒ > > 0,92% qemu-system-arm qemu-system-arm [.] > > tcg_gen_code > > ▒ > > 0,91% qemu-system-arm qemu-system-arm [.] > > qemu_mutex_lock_impl > > ▒ > > 0,75% qemu-system-arm qemu-system-arm [.] > > get_page_addr_code_hostp > > ▒ > > 0,73% qemu-system-arm qemu-system-arm [.] > > tcg_optimize > > ▒ > > 0,71% qemu-system-arm qemu-system-arm [.] > > qemu_mutex_unlock_impl > > ▒ > > 0,69% qemu-system-arm libc-2.32.so [.] > > _int_free > > ▒ > > 0,64% qemu-system-arm qemu-system-arm [.] > > tb_flush_jmp_cache > > ▒ > > 0,63% qemu-system-arm qemu-system-arm [.] > > address_space_ldl_le > > ▒ > > 0,62% qemu-system-arm libpthread-2.32.so [.] > > __lll_lock_wait > > ▒ > > 0,58% qemu-system-arm libpthread-2.32.so [.] > > pthread_cond_wait@@GLIBC_2.3.2 > > ▒ > > 0,52% qemu-system-arm qemu-system-arm [.] > > cpu_reset_interrupt > > ▒ > > 0,52% qemu-system-arm libc-2.32.so [.] > > cfree@GLIBC_2.2.5 > > ▒ > > 0,52% qemu-system-arm qemu-system-arm [.] > > qemu_set_irq > > ▒ > > > > However the absolute time is 3-4 minutes. > > And I don't know where I should start optimization now. > > > > > > > > Am 07.10.20 um 14:02 schrieb Alex Bennée: > > > Thomas Schneider <74cmonty@gmail.com> writes: > > > > > >> Are you referring to this tool? > > >> https://github.com/stefano-garzarella/qemu-boot-time > > >> <https://github.com/stefano-garzarella/qemu-boot-time> > > > No - just plain perf: > > > > > > perf record $QEMU $ARGS > > > > > > Then a "perf report" which will show you the hotspots, for example: > > > > > > 8.92% qemu-system-aar qemu-system-aarch64 [.] helper_lookup_tb_ptr > > > 4.76% qemu-system-aar qemu-system-aarch64 [.] liveness_pass_1 > > > 3.69% qemu-system-aar qemu-system-aarch64 [.] tcg_gen_code > > > 2.95% qemu-system-aar qemu-system-aarch64 [.] qht_lookup_custom > > > 2.93% qemu-system-aar qemu-system-aarch64 [.] tcg_optimize > > > 1.28% qemu-system-aar qemu-system-aarch64 [.] tcg_out_opc.isra.15 > > > 1.09% qemu-system-aar qemu-system-aarch64 [.] get_phys_addr_lpae > > > 1.09% qemu-system-aar [kernel.kallsyms] [k] isolate_freepages_block > > > 1.05% qemu-system-aar qemu-system-aarch64 [.] cpu_get_tb_cpu_state > > > 0.98% qemu-system-aar [kernel.kallsyms] [k] do_syscall_64 > > > 0.94% qemu-system-aar qemu-system-aarch64 [.] tb_lookup_cmp > > > 0.78% qemu-system-aar qemu-system-aarch64 [.] init_ts_info > > > 0.73% qemu-system-aar qemu-system-aarch64 [.] tb_htable_lookup > > > 0.73% qemu-system-aar qemu-system-aarch64 [.] tb_gen_code > > > 0.73% qemu-system-aar qemu-system-aarch64 [.] tlb_set_page_with_attrs > > > > > >> > > >> Am 07.10.2020 um 13:00 schrieb Alex Bennée: > > >>> perf to record your boot > > > > > ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Emulate Rpi with QEMU fails 2020-10-08 7:00 ` Thomas 2020-10-08 21:07 ` Paul Zimmerman @ 2020-10-09 6:20 ` Alex Bennée 1 sibling, 0 replies; 19+ messages in thread From: Alex Bennée @ 2020-10-09 6:20 UTC (permalink / raw) To: Thomas Cc: Peter Maydell, Philippe Mathieu-Daudé, Andrew Baumann, QEMU Developers, qemu-arm, Paul Zimmerman Thomas <74cmonty@gmail.com> writes: > Interesting enough is: my top figure reported by perf is like yours: > > Samples: 6M of event 'cycles:u', Event count (approx.): 1936571734942 > Overhead Command Shared Object Symbol > 7,95% qemu-system-arm qemu-system-arm [.] > helper_lookup_tb_ptr > ◆ > 4,16% qemu-system-arm qemu-system-arm [.] > cpu_get_tb_cpu_state > ▒ Not quite - cpu_get_tb_state should be a fair bit cheaper now since the optimisations to hflags where introduced. > 2,52% qemu-system-arm libpthread-2.32.so [.] > __pthread_mutex_lock > ▒ This indicated there is some lock contention. The QSP profiler can help here. Are you running the latest state of master? -- Alex Bennée ^ permalink raw reply [flat|nested] 19+ messages in thread
end of thread, other threads:[~2020-10-09 6:22 UTC | newest] Thread overview: 19+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2020-10-03 11:45 Emulate Rpi with QEMU fails Thomas 2020-10-04 17:44 ` Alex Bennée 2020-10-04 18:40 ` Peter Maydell 2020-10-05 9:40 ` Alex Bennée 2020-10-05 10:51 ` Thomas Schneider 2020-10-05 22:08 ` Paul Zimmerman 2020-10-06 6:58 ` Thomas Schneider 2020-10-06 7:42 ` Paul Zimmerman 2020-10-06 9:58 ` Alex Bennée 2020-10-07 6:28 ` Thomas 2020-10-07 6:50 ` Paul Zimmerman 2020-10-07 7:27 ` Thomas Schneider 2020-10-07 11:00 ` Alex Bennée 2020-10-07 11:36 ` Thomas Schneider 2020-10-07 12:02 ` Alex Bennée 2020-10-08 7:00 ` Thomas 2020-10-08 21:07 ` Paul Zimmerman 2020-10-09 2:21 ` Paul Zimmerman 2020-10-09 6:20 ` Alex Bennée
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).