[Qemu-devel] QEMU+Linux ARMv7A current state

[Qemu-devel] QEMU+Linux ARMv7A current state

[Peter Crosthwaite]

Hi,

I have done an audit of the ARMv7 boards to see what can boot a
vanilla linux kernel. The basic approach is to build ARM
multi_v7_defconfig kernel and boot QEMU using the DTBs built out by
the kernel. The intersection of what mainline Linux has a DTB for and
what QEMU models is tested. The boards that do/should work include:

    vexpress-a9
    vexpress-a15
    smdkc210 (exynos)
    xilinx-zynq-a9
    cubieboard (allwinner)
    highbank
    midway
    virt

Have I missed anything?

The basic rule is, no source code, config, or DTB edits allowed (i.e.
what is a day0 user of QEMU+Linux going to experience?).

So going board-by-board, these are the issues.

highbank/midway:

The kernel expects a secure monitor to be present which does at-least
two things:

* PSCI
* Some cache-maintainance ops.

QEMU has PSCI support, but the 0.1 command codings specified in the
DTB are different to QEMUs 0.1 encodings. So I switched monitor mode
for the CPUs on (it was conservatively set to off by the original
authors of the QEMU ARM monitor support) and turned PSCI on. I had to
write some patches to remap the PSCI encodings to highbank's custom
ones and then the kernel at least entry-points all 4 cpus. It then
deadlocks on something that I am yet to figure out. So the PSCI SMP
stuff is shelved.

The cache-maintenance is a bigger issue, that stops even a single core
boot. The kernel cache controller driver is doing an smc that QEMU
goes haywire on (as there simply is no firmware to catch the smc). But
caches in QEMU are a nop, so I wrote some monitor firmware that just
erets the smc without action and it works.

I am using Sata (sysbus-ahci) for the boot media which works straight
out of the box.

So single core highbank works with the dummy monitor firmware and the
monitor mode restrictor removed.

cubieboard:

The kernel expects a system level clock controller device that isn't
modelled. A dummy model which hardcodes all registers to a know reset
value and just lets the kernel read and write whatever it wants
resolves. I'm not 100% sure whether this is needed but the kernel
prints multiple errors with many pages of traces without. So it is at
least needed to reduce the dmesg noise.

QEMU cubieboard has no usable storage media, but the real hardware
does have AHCI sata. I added sysbus-ahci at the right place but turns
out the SATA controller has some custom power/clock (not really
sure??) registers specific to this SoC. It sets/clears bits then polls
them back expecting them to change to the other value asynchronously.
The kernel device probe then times-out. So I subclassed sysbus-ahci
and added the missing registers and forced the polled registers to the
"I'm done" state. It works.

I am using meta-sunxi Yocto-layer to build out the allwinner custom
kernel/rootfs etc, and with the clock and Sata changes I get a boot.
But when I change to the unedited kernel+dtb+rootfs I get stuck. RTC
messages are around the point of failure which is not modelled in
QEMU, so that is suspect.

So with a dummy clock-controller and a slightly modified Sata
controller (as a new device) cubieboard progresses, but it is still
stuck on something that looks RTC related.

xilinx-zynq-a9:

The kernel wants to use cpufreq to set the CPU frequency. The kernel
assumes ownership of the CPU clock divider but not the PLLs. QEMU uses
the HW default for the silicon (x26) for the PLL which is incompatible
with the dtb-set CPU frequency. The kernel boot asserts this and
crashes. The ideal solution is to provide some sort of pre-boot
firmware to set it to a compatible value. In real HW, this is the FSBL
bootloader.

TBH I think this is a kernel bug, as since cpufreq can be deconfigured
it is ultimately optional. This means it shouldn't be a showstopper on
a boot. The kernel should just switch off cpufreq and proceed and
behave as if the feature was never configured in the kernel.

There is also a critical bug in the SLCR model (my bad) where write to
registers are ignored. I have a patch.

More reading about the SLCR issues here (thanks to Guenter for getting
the ball rolling here):

https://lists.gnu.org/archive/html/qemu-devel/2015-09/msg03374.html

I have a patch with sets up the needed SLCR values from software but
it needs more work.

The kernel also expect the ADC device which is absent. Thanks to
Guenter who patched this in which is under review:

https://lists.gnu.org/archive/html/qemu-devel/2015-09/msg03375.html

I am using SD card for the rootfs media. SD has a bug where it cannot
handle a backing file of a non-round size. I am just passing the ext3
image in unchanged (so unpadded) to QEMU and QEMU rounds down the
image to the SD block size (512k!!), so I am losing us to 512k off the
end of my ext3 filesystem. I have a hacky patch that changes this to
round-up.

With the minimal firmware, slcr bugfix, Guenters ADC, the SD hack a
vanilla Zynq will be in good shape.

virt:

Virt largely works, but there are no immediately obvious
storage/network options that are supported by the kernel. So I have to
make some exceptions to the kernel config rule, that is, I add:

CONFIG_VIRTIO_PCI=y"
CONFIG_VIRTIO_BLK=y"
CONFIG_VIRTIO_NET=y"

To get disks and network for the virt board.

Could we add these configs to mainline Linux's defconfig? Note, we
don't need to add a DTS for this board, as QEMU's bootloader will
generate and provide it for us.

There is the known highmem PCI issue that is currently under discussion:

https://lists.gnu.org/archive/html/qemu-devel/2015-09/msg06529.html

So aside from the missing VIRTIO kernel configs and highmem bug, virt
is in good shape.

vexpress:

vexpress boots up to rootfs probing, however the only storage media
that seems to be supported is flash, which doesn't have the needed
kernel configs to boot from (not 100% sure on that but I gave up
quickly). What are people who use this board in real hardware doing?
It may just work best with initrd style boots.

exynos:

QEMU models an Exynos skdkc210 but there is only a 310 available in
QEMU dtbs. QEMU exynos however does not model a lot on the board level
so we can largely get away with booting 310 dtb on 210.

Similar to vexpress, the exynos board has a lack of usable boot media.
The board does model USB (EHCI), however the DTB disables it as the
smdk310 board does not support it. So the alternative would be to use
one of the more full-featured exynos DTBs  (with way more peripherals)
from mainline Linux, but the kernel is liable to then probe all sorts
of missing hardware.

Exynos SDHCI support was on list for a few revs and quite some time,
merging it and using SD may solve the problem.

I have not attempted the SMP support.

Unlike all other boards, Exynos is quite consuming of RAM. All other
boards work with Yocto's default of 128MB but Exynos crashes unless
you jack it up. So users need to be careful of the -m option.

realview:

Realview is not tested, but I discuss it here as it is a bit of a red
herring. The kernel has dtb support for the realview-1176 board,
however this is not modelled by QEMU. I creating a Realview variant
with 1176 in QEMU, however the memory maps of all the peripherals were
mismatched.

Turns, out, QEMU is is modelling the realview FPGA emulation platform
which has a different peripheral map to the 1176 realview board
proper. So QEMU doesn't support the realview boards, just the FPGA
emulation platform, for which I can't find a kernel dtb. Should we
upstream one with some dtsi's for the CPU variation supported in QEMU?

So Realview does not work within my testing parameters with no
immediate prospects.

how I am testing:

So this is all powered by the Yocto project (Poky). I got some good
help from Nathan Rossi. I have a poky fork which changes the qemuarm
target to build my mainline (4.2.1) kernel (multi_v7_defconfig +
VIRTIO), all the DTBs and a usable rootfs. You can then specify the
type of machine (from the list above) to yocto's runqemu command. The
command sets up boot media and network automatically.

To use it:

git clone https://github.com/pcrost/poky.git
cd poky
source ./oe-init-build-env
MACHINE=qemuarm bitbake core-image-minimal #this takes a while
QEMUBIN=/path/to/qemu-arm MACHINE_SUBTYPE=virt runqemu qemuarm slirp

The QEMUBIN is optional as Yocto does build QEMU for you, but this
lets you BYO QEMU if you are doing qemu development outside of the
Yocto flow. Change MACHINE_SUBTYPE to one of the supported boards to
see results. I suggest starting with virt. Skip cubieboard, that
assumes my SATA patches. Everything else you will see varying degrees
of success. Pass qemuparams="-m 256" to runqemu for the smdkc200
(Exynos) board. Highbank and Midway will blank screen as the
outstanding issues happen pre-UART. A meaningful logbuf can be pulled
from RAM over the monitor. Useful instructions here:

http://www.wiki.xilinx.com/QEMU-Linux+Kernel+logbuf+Extraction

Some QEMU patches to follow to fix a few of the things mentioned.

We should get this into regression testing. Yocto has some automated
testing features in it's own right that we should be able to leverage.
Yet to investigate (Richard pasted some stuff on an earlier thread).

The exercise should be redone for ARM64, ARMv5 and ARMv6 and ARMv7M,
then other arches (many of which Poky supports).

Regards,
Peter

Re: [Qemu-devel] QEMU+Linux ARMv7A current state

[Peter Maydell]

On 3 October 2015 at 22:31, Peter Crosthwaite
<crosthwaitepeter@gmail.com> wrote:
> Hi,
>
> I have done an audit of the ARMv7 boards to see what can boot a
> vanilla linux kernel. The basic approach is to build ARM
> multi_v7_defconfig kernel and boot QEMU using the DTBs built out by
> the kernel. The intersection of what mainline Linux has a DTB for and
> what QEMU models is tested.

Thanks for doing this; this is an interesting survey of what
we implement.

> The boards that do/should work include:
>
>     vexpress-a9
>     vexpress-a15
>     smdkc210 (exynos)
>     xilinx-zynq-a9
>     cubieboard (allwinner)
>     highbank
>     midway
>     virt

FWIW I have test images for vexpress-a9, vexpress-a15,
cubieboard and virt, though of course these are often
older kernels and probably tweaked configs.

> virt:
>
> Virt largely works, but there are no immediately obvious
> storage/network options that are supported by the kernel. So I have to
> make some exceptions to the kernel config rule, that is, I add:
>
> CONFIG_VIRTIO_PCI=y"
> CONFIG_VIRTIO_BLK=y"
> CONFIG_VIRTIO_NET=y"
>
> To get disks and network for the virt board.
>
> Could we add these configs to mainline Linux's defconfig? Note, we
> don't need to add a DTS for this board, as QEMU's bootloader will
> generate and provide it for us.
>
> There is the known highmem PCI issue that is currently under discussion:
>
> https://lists.gnu.org/archive/html/qemu-devel/2015-09/msg06529.html
>
> So aside from the missing VIRTIO kernel configs and highmem bug, virt
> is in good shape.

Did you build your kernel with LPAE or not? I think an LPAE
config ought to avoid the PCI highmem bug (and it's definitely
what you want for anything that's Cortex-A15 based).

> vexpress:
>
> vexpress boots up to rootfs probing, however the only storage media
> that seems to be supported is flash, which doesn't have the needed
> kernel configs to boot from (not 100% sure on that but I gave up
> quickly). What are people who use this board in real hardware doing?
> It may just work best with initrd style boots.

For real hardware I expect people will be using USB disks.
We don't currently model the USB controller in QEMU, though.
My test images use emulated SD card storage (ie -drive if=sd,file=...).
(Emulated SD card will have lousy performance.)

> realview:
>
> Realview is not tested, but I discuss it here as it is a bit of a red
> herring. The kernel has dtb support for the realview-1176 board,

Actually, "realview-pb1176". The "PB" part is important!

> however this is not modelled by QEMU. I creating a Realview variant
> with 1176 in QEMU, however the memory maps of all the peripherals were
> mismatched.
>
> Turns, out, QEMU is is modelling the realview FPGA emulation platform
> which has a different peripheral map to the 1176 realview board
> proper. So QEMU doesn't support the realview boards, just the FPGA
> emulation platform, for which I can't find a kernel dtb. Should we
> upstream one with some dtsi's for the CPU variation supported in QEMU?

"realview" is not really a very helpful term to use here, because
it's a generic label applied to a whole slew of ARM devboards.
What QEMU models is:
 "realview-eb" -- the RealView Emulation Baseboard with an ARM926
 "realview-eb-mpcore" -- the RealView Emulation Baseboard with an 11MPCore
 "realview-pb-a8" -- the RealView Platform Baseboard for Cortex A8
 "realview-pbx-a9" -- the RealView Platform Baseboard Explore for Cortex A9

The DT in the kernel tree is for the Realview Platform Baseboard
  for 1176, ie PB1176. That's a different board from EB1176, as
you've found. This is why "realview" on its own (or with a CPU
name) is not sufficient to identify a board and why we have those
-eb- and -pb(x)- infixes in our board names :-)

> So Realview does not work within my testing parameters with no
> immediate prospects.

The kernel does support EB, PBA8 and PBX; presumably they haven't
been converted to use DT (yet?).

> how I am testing:
>
> So this is all powered by the Yocto project (Poky). I got some good
> help from Nathan Rossi. I have a poky fork which changes the qemuarm
> target to build my mainline (4.2.1) kernel (multi_v7_defconfig +
> VIRTIO), all the DTBs and a usable rootfs. You can then specify the
> type of machine (from the list above) to yocto's runqemu command. The
> command sets up boot media and network automatically.
>
> To use it:
>
> git clone https://github.com/pcrost/poky.git
> cd poky
> source ./oe-init-build-env
> MACHINE=qemuarm bitbake core-image-minimal #this takes a while
> QEMUBIN=/path/to/qemu-arm MACHINE_SUBTYPE=virt runqemu qemuarm slirp
>
> The QEMUBIN is optional as Yocto does build QEMU for you, but this
> lets you BYO QEMU if you are doing qemu development outside of the
> Yocto flow. Change MACHINE_SUBTYPE to one of the supported boards to
> see results. I suggest starting with virt. Skip cubieboard, that
> assumes my SATA patches. Everything else you will see varying degrees
> of success. Pass qemuparams="-m 256" to runqemu for the smdkc200
> (Exynos) board. Highbank and Midway will blank screen as the
> outstanding issues happen pre-UART. A meaningful logbuf can be pulled
> from RAM over the monitor. Useful instructions here:
>
> http://www.wiki.xilinx.com/QEMU-Linux+Kernel+logbuf+Extraction
>
> Some QEMU patches to follow to fix a few of the things mentioned.
>
> We should get this into regression testing. Yocto has some automated
> testing features in it's own right that we should be able to leverage.
> Yet to investigate (Richard pasted some stuff on an earlier thread).

Nice work.

thanks
-- PMM

Re: [Qemu-devel] QEMU+Linux ARMv7A current state

[Peter Crosthwaite]

On Sat, Oct 3, 2015 at 2:51 PM, Peter Maydell <peter.maydell@linaro.org> wrote:
> On 3 October 2015 at 22:31, Peter Crosthwaite
> <crosthwaitepeter@gmail.com> wrote:
>> Hi,
>>
>> I have done an audit of the ARMv7 boards to see what can boot a
>> vanilla linux kernel. The basic approach is to build ARM
>> multi_v7_defconfig kernel and boot QEMU using the DTBs built out by
>> the kernel. The intersection of what mainline Linux has a DTB for and
>> what QEMU models is tested.
>
> Thanks for doing this; this is an interesting survey of what
> we implement.
>
>> The boards that do/should work include:
>>
>>     vexpress-a9
>>     vexpress-a15
>>     smdkc210 (exynos)
>>     xilinx-zynq-a9
>>     cubieboard (allwinner)
>>     highbank
>>     midway
>>     virt
>
> FWIW I have test images for vexpress-a9, vexpress-a15,
> cubieboard and virt, though of course these are often
> older kernels and probably tweaked configs.
>

Yes. I am avoiding tweaked configs, as I am trying to be
representative of someone trying to RYO Linux without QEMU
implementation knowledge.

>> virt:
>>
>> Virt largely works, but there are no immediately obvious
>> storage/network options that are supported by the kernel. So I have to
>> make some exceptions to the kernel config rule, that is, I add:
>>
>> CONFIG_VIRTIO_PCI=y"
>> CONFIG_VIRTIO_BLK=y"
>> CONFIG_VIRTIO_NET=y"
>>
>> To get disks and network for the virt board.
>>
>> Could we add these configs to mainline Linux's defconfig? Note, we
>> don't need to add a DTS for this board, as QEMU's bootloader will
>> generate and provide it for us.
>>
>> There is the known highmem PCI issue that is currently under discussion:
>>
>> https://lists.gnu.org/archive/html/qemu-devel/2015-09/msg06529.html
>>
>> So aside from the missing VIRTIO kernel configs and highmem bug, virt
>> is in good shape.
>
> Did you build your kernel with LPAE or not? I think an LPAE
> config ought to avoid the PCI highmem bug (and it's definitely
> what you want for anything that's Cortex-A15 based).
>

No. Although any missing configs I consider to be our problem, or a
bug in the multi_v7_defconfig in upstream Linux. That defconfig really
should work for us.

>> vexpress:
>>
>> vexpress boots up to rootfs probing, however the only storage media
>> that seems to be supported is flash, which doesn't have the needed
>> kernel configs to boot from (not 100% sure on that but I gave up
>> quickly). What are people who use this board in real hardware doing?
>> It may just work best with initrd style boots.
>
> For real hardware I expect people will be using USB disks.
> We don't currently model the USB controller in QEMU, though.

Looks like a custom job too. Not much chance here unless this is a
rebadging of one of the existing HCIs?

> My test images use emulated SD card storage (ie -drive if=sd,file=...).
> (Emulated SD card will have lousy performance.)
>

Ok that's a good plan A. Emulated SDHCI on Zynq does just fine for
this level of testing. I missed it as my options search was for "SD"
rather than "MMC".

>> realview:
>>
>> Realview is not tested, but I discuss it here as it is a bit of a red
>> herring. The kernel has dtb support for the realview-1176 board,
>
> Actually, "realview-pb1176". The "PB" part is important!
>
>> however this is not modelled by QEMU. I creating a Realview variant
>> with 1176 in QEMU, however the memory maps of all the peripherals were
>> mismatched.
>>
>> Turns, out, QEMU is is modelling the realview FPGA emulation platform
>> which has a different peripheral map to the 1176 realview board
>> proper. So QEMU doesn't support the realview boards, just the FPGA
>> emulation platform, for which I can't find a kernel dtb. Should we
>> upstream one with some dtsi's for the CPU variation supported in QEMU?
>
> "realview" is not really a very helpful term to use here, because
> it's a generic label applied to a whole slew of ARM devboards.
> What QEMU models is:
>  "realview-eb" -- the RealView Emulation Baseboard with an ARM926
>  "realview-eb-mpcore" -- the RealView Emulation Baseboard with an 11MPCore
>  "realview-pb-a8" -- the RealView Platform Baseboard for Cortex A8
>  "realview-pbx-a9" -- the RealView Platform Baseboard Explore for Cortex A9
>
> The DT in the kernel tree is for the Realview Platform Baseboard
>   for 1176, ie PB1176. That's a different board from EB1176, as
> you've found. This is why "realview" on its own (or with a CPU
> name) is not sufficient to identify a board and why we have those
> -eb- and -pb(x)- infixes in our board names :-)
>

So does this mean that the EB memory maps are consistent but perhaps
not the PB? Can we reliably gen all combinations with EB vs PB with
just two memory maps, then add 1176 both ways?

Regards,
Peter

>> So Realview does not work within my testing parameters with no
>> immediate prospects.
>
> The kernel does support EB, PBA8 and PBX; presumably they haven't
> been converted to use DT (yet?).
>
>> how I am testing:
>>
>> So this is all powered by the Yocto project (Poky). I got some good
>> help from Nathan Rossi. I have a poky fork which changes the qemuarm
>> target to build my mainline (4.2.1) kernel (multi_v7_defconfig +
>> VIRTIO), all the DTBs and a usable rootfs. You can then specify the
>> type of machine (from the list above) to yocto's runqemu command. The
>> command sets up boot media and network automatically.
>>
>> To use it:
>>
>> git clone https://github.com/pcrost/poky.git
>> cd poky
>> source ./oe-init-build-env
>> MACHINE=qemuarm bitbake core-image-minimal #this takes a while
>> QEMUBIN=/path/to/qemu-arm MACHINE_SUBTYPE=virt runqemu qemuarm slirp
>>
>> The QEMUBIN is optional as Yocto does build QEMU for you, but this
>> lets you BYO QEMU if you are doing qemu development outside of the
>> Yocto flow. Change MACHINE_SUBTYPE to one of the supported boards to
>> see results. I suggest starting with virt. Skip cubieboard, that
>> assumes my SATA patches. Everything else you will see varying degrees
>> of success. Pass qemuparams="-m 256" to runqemu for the smdkc200
>> (Exynos) board. Highbank and Midway will blank screen as the
>> outstanding issues happen pre-UART. A meaningful logbuf can be pulled
>> from RAM over the monitor. Useful instructions here:
>>
>> http://www.wiki.xilinx.com/QEMU-Linux+Kernel+logbuf+Extraction
>>
>> Some QEMU patches to follow to fix a few of the things mentioned.
>>
>> We should get this into regression testing. Yocto has some automated
>> testing features in it's own right that we should be able to leverage.
>> Yet to investigate (Richard pasted some stuff on an earlier thread).
>
> Nice work.
>
> thanks
> -- PMM

Re: [Qemu-devel] QEMU+Linux ARMv7A current state

[Peter Crosthwaite]

On Sat, Oct 3, 2015 at 3:14 PM, Peter Crosthwaite
<crosthwaitepeter@gmail.com> wrote:
> On Sat, Oct 3, 2015 at 2:51 PM, Peter Maydell <peter.maydell@linaro.org> wrote:
>> On 3 October 2015 at 22:31, Peter Crosthwaite
>> <crosthwaitepeter@gmail.com> wrote:
>>> Hi,

>>> vexpress:
>>>
>>> vexpress boots up to rootfs probing, however the only storage media
>>> that seems to be supported is flash, which doesn't have the needed
>>> kernel configs to boot from (not 100% sure on that but I gave up
>>> quickly). What are people who use this board in real hardware doing?
>>> It may just work best with initrd style boots.
>>
>> For real hardware I expect people will be using USB disks.
>> We don't currently model the USB controller in QEMU, though.
>
> Looks like a custom job too. Not much chance here unless this is a
> rebadging of one of the existing HCIs?
>
>> My test images use emulated SD card storage (ie -drive if=sd,file=...).
>> (Emulated SD card will have lousy performance.)
>>
>
> Ok that's a good plan A. Emulated SDHCI on Zynq does just fine for
> this level of testing. I missed it as my options search was for "SD"
> rather than "MMC".
>

Ok it works. It suffers from the same 512k truncation bug that I
described for Zynq so it needs that workaround, but otherwise it is in
good shape. Vexpress is done and dusted (Poky diff):

diff --git a/scripts/runqemu-internal b/scripts/runqemu-internal
index a777d8e..a92c1b7 100755
--- a/scripts/runqemu-internal
+++ b/scripts/runqemu-internal
@@ -374,8 +374,10 @@ if [ "$MACHINE" = "qemuarm" -o "$MACHINE" =
"qemuarmv6" -o "$MACHINE" = "qemuarm
         ;;
     vexpress-a9)
         KERNCMDLINE_CONSOLE="ttyAMA0"
+        KERNCMDLINE_ROOTDEV="/dev/mmcblk0"
         QEMU_UI_OPTIONS="-nographic -serial mon:stdio"
         QEMU_OPTIONS="-dtb ${KERNEL}-vexpress-v2p-ca9.dtb"
+        QEMU_DRIVE_OPTIONS=",if=sd"
         ;;
      vexpress-a15)
         KERNCMDLINE_CONSOLE="ttyAMA0"

Regards,
Peter

Re: [Qemu-devel] QEMU+Linux ARMv7A current state

[Guenter Roeck]

On 10/03/2015 02:31 PM, Peter Crosthwaite wrote:
> Hi,
>
> I have done an audit of the ARMv7 boards to see what can boot a
> vanilla linux kernel. The basic approach is to build ARM
> multi_v7_defconfig kernel and boot QEMU using the DTBs built out by
> the kernel. The intersection of what mainline Linux has a DTB for and
> what QEMU models is tested. The boards that do/should work include:
>
>      vexpress-a9
>      vexpress-a15
>      smdkc210 (exynos)
>      xilinx-zynq-a9
>      cubieboard (allwinner)
>      highbank
>      midway
>      virt
>
> Have I missed anything?
>
> The basic rule is, no source code, config, or DTB edits allowed (i.e.
> what is a day0 user of QEMU+Linux going to experience?).
>
> So going board-by-board, these are the issues.
>
> highbank/midway:
>
> The kernel expects a secure monitor to be present which does at-least
> two things:
>
> * PSCI
> * Some cache-maintainance ops.
>
> QEMU has PSCI support, but the 0.1 command codings specified in the
> DTB are different to QEMUs 0.1 encodings. So I switched monitor mode
> for the CPUs on (it was conservatively set to off by the original
> authors of the QEMU ARM monitor support) and turned PSCI on. I had to
> write some patches to remap the PSCI encodings to highbank's custom
> ones and then the kernel at least entry-points all 4 cpus. It then
> deadlocks on something that I am yet to figure out. So the PSCI SMP
> stuff is shelved.
>
> The cache-maintenance is a bigger issue, that stops even a single core
> boot. The kernel cache controller driver is doing an smc that QEMU
> goes haywire on (as there simply is no firmware to catch the smc). But
> caches in QEMU are a nop, so I wrote some monitor firmware that just
> erets the smc without action and it works.
>
> I am using Sata (sysbus-ahci) for the boot media which works straight
> out of the box.
>
> So single core highbank works with the dummy monitor firmware and the
> monitor mode restrictor removed.
>
I have highbank working with multi_v7_defconfig. It generates a warning, though.

WARNING: CPU: 0 PID: 0 at arch/arm/mach-highbank/highbank.c:60 highbank_l2c310_write_sec+0x34/0x58()
[    0.000000] Highbank L2C310: ignoring write to reg 0x104

I think this is a kernel problem.

It also generates an SMC request.

SMC cmd 0x102 val 0x1

My "fix" for that is to simply log and ignore all SMC requests in qemu.
Not really feasible for production qemu, but after all I am testing the kernel,
not qemu. That also helped for various other HW. Maybe your monitor firmware
can help me get rid of that.

On top of that, I had to enable EL3. I don't remember why, just that it was
needed.

> cubieboard:
>
> The kernel expects a system level clock controller device that isn't
> modelled. A dummy model which hardcodes all registers to a know reset
> value and just lets the kernel read and write whatever it wants
> resolves. I'm not 100% sure whether this is needed but the kernel
> prints multiple errors with many pages of traces without. So it is at
> least needed to reduce the dmesg noise.
>
> QEMU cubieboard has no usable storage media, but the real hardware
> does have AHCI sata. I added sysbus-ahci at the right place but turns
> out the SATA controller has some custom power/clock (not really
> sure??) registers specific to this SoC. It sets/clears bits then polls
> them back expecting them to change to the other value asynchronously.
> The kernel device probe then times-out. So I subclassed sysbus-ahci
> and added the missing registers and forced the polled registers to the
> "I'm done" state. It works.
>
> I am using meta-sunxi Yocto-layer to build out the allwinner custom
> kernel/rootfs etc, and with the clock and Sata changes I get a boot.
> But when I change to the unedited kernel+dtb+rootfs I get stuck. RTC
> messages are around the point of failure which is not modelled in
> QEMU, so that is suspect.
>
> So with a dummy clock-controller and a slightly modified Sata
> controller (as a new device) cubieboard progresses, but it is still
> stuck on something that looks RTC related.
>
> xilinx-zynq-a9:
>
> The kernel wants to use cpufreq to set the CPU frequency. The kernel
> assumes ownership of the CPU clock divider but not the PLLs. QEMU uses
> the HW default for the silicon (x26) for the PLL which is incompatible
> with the dtb-set CPU frequency. The kernel boot asserts this and
> crashes. The ideal solution is to provide some sort of pre-boot
> firmware to set it to a compatible value. In real HW, this is the FSBL
> bootloader.
>
> TBH I think this is a kernel bug, as since cpufreq can be deconfigured
> it is ultimately optional. This means it shouldn't be a showstopper on
> a boot. The kernel should just switch off cpufreq and proceed and
> behave as if the feature was never configured in the kernel.
>
> There is also a critical bug in the SLCR model (my bad) where write to
> registers are ignored. I have a patch.
>
> More reading about the SLCR issues here (thanks to Guenter for getting
> the ball rolling here):
>
> https://lists.gnu.org/archive/html/qemu-devel/2015-09/msg03374.html
>
Seemed to be going nowhere, though :-(.

> I have a patch with sets up the needed SLCR values from software but
> it needs more work.
>
> The kernel also expect the ADC device which is absent. Thanks to
> Guenter who patched this in which is under review:
>
> https://lists.gnu.org/archive/html/qemu-devel/2015-09/msg03375.html
>
Still working on the review comments.

> I am using SD card for the rootfs media. SD has a bug where it cannot
> handle a backing file of a non-round size. I am just passing the ext3
> image in unchanged (so unpadded) to QEMU and QEMU rounds down the
> image to the SD block size (512k!!), so I am losing us to 512k off the
> end of my ext3 filesystem. I have a hacky patch that changes this to
> round-up.
>
> With the minimal firmware, slcr bugfix, Guenters ADC, the SD hack a
> vanilla Zynq will be in good shape.
>
> virt:
>
> Virt largely works, but there are no immediately obvious
> storage/network options that are supported by the kernel. So I have to
> make some exceptions to the kernel config rule, that is, I add:
>
> CONFIG_VIRTIO_PCI=y"
> CONFIG_VIRTIO_BLK=y"
> CONFIG_VIRTIO_NET=y"
>
> To get disks and network for the virt board.
>
> Could we add these configs to mainline Linux's defconfig? Note, we
> don't need to add a DTS for this board, as QEMU's bootloader will
> generate and provide it for us.
>
> There is the known highmem PCI issue that is currently under discussion:
>
> https://lists.gnu.org/archive/html/qemu-devel/2015-09/msg06529.html
>
> So aside from the missing VIRTIO kernel configs and highmem bug, virt
> is in good shape.
>
> vexpress:
>
> vexpress boots up to rootfs probing, however the only storage media
> that seems to be supported is flash, which doesn't have the needed
> kernel configs to boot from (not 100% sure on that but I gave up
> quickly). What are people who use this board in real hardware doing?
> It may just work best with initrd style boots.
>
I use it with
	-drive file=rootfs.ext3,format=raw,if=sd \
	-append "root=/dev/mmcblk0 ..."

with multi_v7_defconfig and vexpress-v2p-ca9.dtb. This works for both
vexpress-a9 and vexpress-a15 (with vexpress-v2p-ca15-tc1.dtb).

> exynos:
>
> QEMU models an Exynos skdkc210 but there is only a 310 available in
> QEMU dtbs. QEMU exynos however does not model a lot on the board level
> so we can largely get away with booting 310 dtb on 210.
>
I needed a couple of fixes to get the smdkc210 machine to work with
exynos4210-smdkv310.dtb.

- CMU support was submitted a long time ago by Maksim Kozlov, but never
   accepted. I took the code and merged the absolutely necessary parts of
   (I dropped clock support).
- exynos4210_mct needs a one-line bug fix to initialize a timer before
   it is used.
- dma support for exynos4 must be enabled (pl330 must be instantiated).
- If I recall correctly, smdkc210 also needed SMC support.

I have to change the configuration file, though. QEMU currently does not
implement exynos cpuidle, so I have to disable CONFIG_CPU_IDLE and
CONFIG_ARM_EXYNOS_CPUIDLE.

I test both multi_v7_defconfig and exynos_defconfig with -M smdkc210.

> Similar to vexpress, the exynos board has a lack of usable boot media.
> The board does model USB (EHCI), however the DTB disables it as the
> smdk310 board does not support it. So the alternative would be to use
> one of the more full-featured exynos DTBs  (with way more peripherals)
> from mainline Linux, but the kernel is liable to then probe all sorts
> of missing hardware.
>
> Exynos SDHCI support was on list for a few revs and quite some time,
> merging it and using SD may solve the problem.
>
> I have not attempted the SMP support.
>
I run smdkc210 with "-smp 2". If I recall correctly, this is mandatory,
but I may be wrong.

> Unlike all other boards, Exynos is quite consuming of RAM. All other
> boards work with Yocto's default of 128MB but Exynos crashes unless
> you jack it up. So users need to be careful of the -m option.
>
> realview:
>
> Realview is not tested, but I discuss it here as it is a bit of a red
> herring. The kernel has dtb support for the realview-1176 board,
> however this is not modelled by QEMU. I creating a Realview variant
> with 1176 in QEMU, however the memory maps of all the peripherals were
> mismatched.
>
> Turns, out, QEMU is is modelling the realview FPGA emulation platform
> which has a different peripheral map to the 1176 realview board
> proper. So QEMU doesn't support the realview boards, just the FPGA
> emulation platform, for which I can't find a kernel dtb. Should we
> upstream one with some dtsi's for the CPU variation supported in QEMU?
>
> So Realview does not work within my testing parameters with no
> immediate prospects.
>
I have realview-pb-a8, realview-eb-mpcore, and realview-eb working,
but not with a default configuration.

> how I am testing:
>
> So this is all powered by the Yocto project (Poky). I got some good
> help from Nathan Rossi. I have a poky fork which changes the qemuarm
> target to build my mainline (4.2.1) kernel (multi_v7_defconfig +
> VIRTIO), all the DTBs and a usable rootfs. You can then specify the
> type of machine (from the list above) to yocto's runqemu command. The
> command sets up boot media and network automatically.
>
> To use it:
>
> git clone https://github.com/pcrost/poky.git
> cd poky
> source ./oe-init-build-env
> MACHINE=qemuarm bitbake core-image-minimal #this takes a while
> QEMUBIN=/path/to/qemu-arm MACHINE_SUBTYPE=virt runqemu qemuarm slirp
>
Just for reference, my code is at github.

https://github.com/groeck/linux-build-test
https://github.com/groeck/qemu

Guenter

Re: [Qemu-devel] QEMU+Linux ARMv7A current state

[Peter Crosthwaite]

On Sat, Oct 3, 2015 at 5:21 PM, Guenter Roeck <linux@roeck-us.net> wrote:
> On 10/03/2015 02:31 PM, Peter Crosthwaite wrote:
>>
>> Hi,
>>
>> I have done an audit of the ARMv7 boards to see what can boot a
>> vanilla linux kernel. The basic approach is to build ARM
>> multi_v7_defconfig kernel and boot QEMU using the DTBs built out by
>> the kernel. The intersection of what mainline Linux has a DTB for and
>> what QEMU models is tested. The boards that do/should work include:
>>
>>      vexpress-a9
>>      vexpress-a15
>>      smdkc210 (exynos)
>>      xilinx-zynq-a9
>>      cubieboard (allwinner)
>>      highbank
>>      midway
>>      virt
>>
>> Have I missed anything?
>>
>> The basic rule is, no source code, config, or DTB edits allowed (i.e.
>> what is a day0 user of QEMU+Linux going to experience?).
>>
>> So going board-by-board, these are the issues.
>>
>> highbank/midway:
>>
>> The kernel expects a secure monitor to be present which does at-least
>> two things:
>>
>> * PSCI
>> * Some cache-maintainance ops.
>>
>> QEMU has PSCI support, but the 0.1 command codings specified in the
>> DTB are different to QEMUs 0.1 encodings. So I switched monitor mode
>> for the CPUs on (it was conservatively set to off by the original
>> authors of the QEMU ARM monitor support) and turned PSCI on. I had to
>> write some patches to remap the PSCI encodings to highbank's custom
>> ones and then the kernel at least entry-points all 4 cpus. It then
>> deadlocks on something that I am yet to figure out. So the PSCI SMP
>> stuff is shelved.
>>
>> The cache-maintenance is a bigger issue, that stops even a single core
>> boot. The kernel cache controller driver is doing an smc that QEMU
>> goes haywire on (as there simply is no firmware to catch the smc). But
>> caches in QEMU are a nop, so I wrote some monitor firmware that just
>> erets the smc without action and it works.
>>
>> I am using Sata (sysbus-ahci) for the boot media which works straight
>> out of the box.
>>
>> So single core highbank works with the dummy monitor firmware and the
>> monitor mode restrictor removed.
>>
> I have highbank working with multi_v7_defconfig. It generates a warning,
> though.
>
> WARNING: CPU: 0 PID: 0 at arch/arm/mach-highbank/highbank.c:60
> highbank_l2c310_write_sec+0x34/0x58()
> [    0.000000] Highbank L2C310: ignoring write to reg 0x104
>

Yes I am ignoring this.

> I think this is a kernel problem.
>
> It also generates an SMC request.
>
> SMC cmd 0x102 val 0x1
>
> My "fix" for that is to simply log and ignore all SMC requests in qemu.
> Not really feasible for production qemu, but after all I am testing the
> kernel,
> not qemu. That also helped for various other HW. Maybe your monitor firmware
> can help me get rid of that.
>

Yes. My change doesn't touch target-arm and keeps it all in highbank.c.

> On top of that, I had to enable EL3. I don't remember why, just that it was
> needed.
>

To bring your SMC printf into play most probably. It would just undef
instruction otherwise. I have the same change in my tree.

>
>> cubieboard:
>>
>> The kernel expects a system level clock controller device that isn't
>> modelled. A dummy model which hardcodes all registers to a know reset
>> value and just lets the kernel read and write whatever it wants
>> resolves. I'm not 100% sure whether this is needed but the kernel
>> prints multiple errors with many pages of traces without. So it is at
>> least needed to reduce the dmesg noise.
>>
>> QEMU cubieboard has no usable storage media, but the real hardware
>> does have AHCI sata. I added sysbus-ahci at the right place but turns
>> out the SATA controller has some custom power/clock (not really
>> sure??) registers specific to this SoC. It sets/clears bits then polls
>> them back expecting them to change to the other value asynchronously.
>> The kernel device probe then times-out. So I subclassed sysbus-ahci
>> and added the missing registers and forced the polled registers to the
>> "I'm done" state. It works.
>>
>> I am using meta-sunxi Yocto-layer to build out the allwinner custom
>> kernel/rootfs etc, and with the clock and Sata changes I get a boot.
>> But when I change to the unedited kernel+dtb+rootfs I get stuck. RTC
>> messages are around the point of failure which is not modelled in
>> QEMU, so that is suspect.
>>
>> So with a dummy clock-controller and a slightly modified Sata
>> controller (as a new device) cubieboard progresses, but it is still
>> stuck on something that looks RTC related.
>>
>> xilinx-zynq-a9:
>>
>> The kernel wants to use cpufreq to set the CPU frequency. The kernel
>> assumes ownership of the CPU clock divider but not the PLLs. QEMU uses
>> the HW default for the silicon (x26) for the PLL which is incompatible
>> with the dtb-set CPU frequency. The kernel boot asserts this and
>> crashes. The ideal solution is to provide some sort of pre-boot
>> firmware to set it to a compatible value. In real HW, this is the FSBL
>> bootloader.
>>
>> TBH I think this is a kernel bug, as since cpufreq can be deconfigured
>> it is ultimately optional. This means it shouldn't be a showstopper on
>> a boot. The kernel should just switch off cpufreq and proceed and
>> behave as if the feature was never configured in the kernel.
>>
>> There is also a critical bug in the SLCR model (my bad) where write to
>> registers are ignored. I have a patch.
>>
>> More reading about the SLCR issues here (thanks to Guenter for getting
>> the ball rolling here):
>>
>> https://lists.gnu.org/archive/html/qemu-devel/2015-09/msg03374.html
>>
> Seemed to be going nowhere, though :-(.
>

I'll get there :)

>> I have a patch with sets up the needed SLCR values from software but
>> it needs more work.
>>
>> The kernel also expect the ADC device which is absent. Thanks to
>> Guenter who patched this in which is under review:
>>
>> https://lists.gnu.org/archive/html/qemu-devel/2015-09/msg03375.html
>>
> Still working on the review comments.
>
>
>> I am using SD card for the rootfs media. SD has a bug where it cannot
>> handle a backing file of a non-round size. I am just passing the ext3
>> image in unchanged (so unpadded) to QEMU and QEMU rounds down the
>> image to the SD block size (512k!!), so I am losing us to 512k off the
>> end of my ext3 filesystem. I have a hacky patch that changes this to
>> round-up.
>>
>> With the minimal firmware, slcr bugfix, Guenters ADC, the SD hack a
>> vanilla Zynq will be in good shape.
>>
>> virt:
>>
>> Virt largely works, but there are no immediately obvious
>> storage/network options that are supported by the kernel. So I have to
>> make some exceptions to the kernel config rule, that is, I add:
>>
>> CONFIG_VIRTIO_PCI=y"
>> CONFIG_VIRTIO_BLK=y"
>> CONFIG_VIRTIO_NET=y"
>>
>> To get disks and network for the virt board.
>>
>> Could we add these configs to mainline Linux's defconfig? Note, we
>> don't need to add a DTS for this board, as QEMU's bootloader will
>> generate and provide it for us.
>>
>> There is the known highmem PCI issue that is currently under discussion:
>>
>> https://lists.gnu.org/archive/html/qemu-devel/2015-09/msg06529.html
>>
>> So aside from the missing VIRTIO kernel configs and highmem bug, virt
>> is in good shape.
>>
>> vexpress:
>>
>> vexpress boots up to rootfs probing, however the only storage media
>> that seems to be supported is flash, which doesn't have the needed
>> kernel configs to boot from (not 100% sure on that but I gave up
>> quickly). What are people who use this board in real hardware doing?
>> It may just work best with initrd style boots.
>>
> I use it with
>         -drive file=rootfs.ext3,format=raw,if=sd \
>         -append "root=/dev/mmcblk0 ..."
>

I now have the same. Did you run into the 512k bug? or are your
padding your rootfs images?

> with multi_v7_defconfig and vexpress-v2p-ca9.dtb. This works for both
> vexpress-a9 and vexpress-a15 (with vexpress-v2p-ca15-tc1.dtb).
>
>> exynos:
>>
>> QEMU models an Exynos skdkc210 but there is only a 310 available in
>> QEMU dtbs. QEMU exynos however does not model a lot on the board level
>> so we can largely get away with booting 310 dtb on 210.
>>
> I needed a couple of fixes to get the smdkc210 machine to work with
> exynos4210-smdkv310.dtb.
>
> - CMU support was submitted a long time ago by Maksim Kozlov, but never
>   accepted. I took the code and merged the absolutely necessary parts of
>   (I dropped clock support).
> - exynos4210_mct needs a one-line bug fix to initialize a timer before
>   it is used.
> - dma support for exynos4 must be enabled (pl330 must be instantiated).
> - If I recall correctly, smdkc210 also needed SMC support.
>
> I have to change the configuration file, though. QEMU currently does not
> implement exynos cpuidle, so I have to disable CONFIG_CPU_IDLE and
> CONFIG_ARM_EXYNOS_CPUIDLE.
>
> I test both multi_v7_defconfig and exynos_defconfig with -M smdkc210.
>

Are your changes only needed for the exynos_defconfig?

>> Similar to vexpress, the exynos board has a lack of usable boot media.
>> The board does model USB (EHCI), however the DTB disables it as the
>> smdk310 board does not support it. So the alternative would be to use
>> one of the more full-featured exynos DTBs  (with way more peripherals)
>> from mainline Linux, but the kernel is liable to then probe all sorts
>> of missing hardware.
>>
>> Exynos SDHCI support was on list for a few revs and quite some time,
>> merging it and using SD may solve the problem.
>>
>> I have not attempted the SMP support.
>>
> I run smdkc210 with "-smp 2". If I recall correctly, this is mandatory,
> but I may be wrong.
>

Makes sense. I saw a warning to that effect.

>> Unlike all other boards, Exynos is quite consuming of RAM. All other
>> boards work with Yocto's default of 128MB but Exynos crashes unless
>> you jack it up. So users need to be careful of the -m option.
>>
>> realview:
>>
>> Realview is not tested, but I discuss it here as it is a bit of a red
>> herring. The kernel has dtb support for the realview-1176 board,
>> however this is not modelled by QEMU. I creating a Realview variant
>> with 1176 in QEMU, however the memory maps of all the peripherals were
>> mismatched.
>>
>> Turns, out, QEMU is is modelling the realview FPGA emulation platform
>> which has a different peripheral map to the 1176 realview board
>> proper. So QEMU doesn't support the realview boards, just the FPGA
>> emulation platform, for which I can't find a kernel dtb. Should we
>> upstream one with some dtsi's for the CPU variation supported in QEMU?
>>
>> So Realview does not work within my testing parameters with no
>> immediate prospects.
>>
> I have realview-pb-a8, realview-eb-mpcore, and realview-eb working,
> but not with a default configuration.
>

So if it can be reduced to a "ARCH=arm make some_defconfig; make -j;"
I think we should go for it. Your repo has what looks like
QEMU-specific realview defconfigs. Upstream has a realview_defconfig,
is this likely to work or are there issues?

>> how I am testing:
>>
>> So this is all powered by the Yocto project (Poky). I got some good
>> help from Nathan Rossi. I have a poky fork which changes the qemuarm
>> target to build my mainline (4.2.1) kernel (multi_v7_defconfig +
>> VIRTIO), all the DTBs and a usable rootfs. You can then specify the
>> type of machine (from the list above) to yocto's runqemu command. The
>> command sets up boot media and network automatically.
>>
>> To use it:
>>
>> git clone https://github.com/pcrost/poky.git
>> cd poky
>> source ./oe-init-build-env
>> MACHINE=qemuarm bitbake core-image-minimal #this takes a while
>> QEMUBIN=/path/to/qemu-arm MACHINE_SUBTYPE=virt runqemu qemuarm slirp
>>
> Just for reference, my code is at github.
>
> https://github.com/groeck/linux-build-test
> https://github.com/groeck/qemu
>

There's some cool stuff in your QEMU branch. I made some high level
comments on the github. The IMX timer will need a bit of a rebase
following Jean-Christophe's refactoring.

Regards,
Peter

> Guenter
>

Re: [Qemu-devel] QEMU+Linux ARMv7A current state

[Peter Maydell]

On 3 October 2015 at 23:14, Peter Crosthwaite
<crosthwaitepeter@gmail.com> wrote:
> On Sat, Oct 3, 2015 at 2:51 PM, Peter Maydell <peter.maydell@linaro.org> wrote:
>> Did you build your kernel with LPAE or not? I think an LPAE
>> config ought to avoid the PCI highmem bug (and it's definitely
>> what you want for anything that's Cortex-A15 based).
>>
>
> No. Although any missing configs I consider to be our problem, or a
> bug in the multi_v7_defconfig in upstream Linux. That defconfig really
> should work for us.

IIRC you can't build a kernel for both LPAE and non-LPAE at
once -- the two are mutually exclusive.

>> For real hardware I expect people will be using USB disks.
>> We don't currently model the USB controller in QEMU, though.
>
> Looks like a custom job too. Not much chance here unless this is a
> rebadging of one of the existing HCIs?

It's a Philips ISP1761. I think it's EHCI + USB OTG + probably
some minor registers. It could be modelled in theory...

>> "realview" is not really a very helpful term to use here, because
>> it's a generic label applied to a whole slew of ARM devboards.
>> What QEMU models is:
>>  "realview-eb" -- the RealView Emulation Baseboard with an ARM926
>>  "realview-eb-mpcore" -- the RealView Emulation Baseboard with an 11MPCore
>>  "realview-pb-a8" -- the RealView Platform Baseboard for Cortex A8
>>  "realview-pbx-a9" -- the RealView Platform Baseboard Explore for Cortex A9
>>
>> The DT in the kernel tree is for the Realview Platform Baseboard
>>   for 1176, ie PB1176. That's a different board from EB1176, as
>> you've found. This is why "realview" on its own (or with a CPU
>> name) is not sufficient to identify a board and why we have those
>> -eb- and -pb(x)- infixes in our board names :-)
>>
>
> So does this mean that the EB memory maps are consistent but perhaps
> not the PB? Can we reliably gen all combinations with EB vs PB with
> just two memory maps, then add 1176 both ways?

I think the PB1176 looks more like the PB926 (which we model as
"versatilepb") than the PBA8 or PBXA9 -- for instance the latter
two have a completely different PCI controller. You'd need to
cross-check all the board manuals to be sure.

Regardless, rather than implementing a model of yet another
old ARM devboard nobody really cares about, it seems to me
that the effort would be better spent in converting the kernel
support for the boards we already model to use device tree...

thanks
-- PMM

Re: [Qemu-devel] QEMU+Linux ARMv7A current state

[Beniamino Galvani]

On Sat, Oct 03, 2015 at 02:31:08PM -0700, Peter Crosthwaite wrote:
> QEMU cubieboard has no usable storage media, but the real hardware
> does have AHCI sata. I added sysbus-ahci at the right place but turns
> out the SATA controller has some custom power/clock (not really
> sure??) registers specific to this SoC. It sets/clears bits then polls
> them back expecting them to change to the other value asynchronously.
> The kernel device probe then times-out. So I subclassed sysbus-ahci
> and added the missing registers and forced the polled registers to the
> "I'm done" state. It works.

Cool, are you going to submit patches for this?

> I am using meta-sunxi Yocto-layer to build out the allwinner custom
> kernel/rootfs etc, and with the clock and Sata changes I get a boot.
> But when I change to the unedited kernel+dtb+rootfs I get stuck. RTC
> messages are around the point of failure which is not modelled in
> QEMU, so that is suspect.

I don't know, this needs some investigation; on my side a recent
multi_v7_defconfig kernel, unmodified sun4i-a10-cubieboard.dtb and a
rootfs built with buildroot mounted through NFS work just fine, with
the mentioned warnings regarding clk registers and also these:

Ignoring attempt to switch CPSR_A flag from non-secure world with SCR.AW bit clear
Ignoring attempt to switch CPSR_F flag from non-secure world with SCR.FW bit clear

which probably would be solved by setting the property 'has_el3' of
the CPU to false before realization.

Beniamino

Re: [Qemu-devel] QEMU+Linux ARMv7A current state

[Guenter Roeck]

On 10/04/2015 12:56 PM, Beniamino Galvani wrote:
> On Sat, Oct 03, 2015 at 02:31:08PM -0700, Peter Crosthwaite wrote:
>> QEMU cubieboard has no usable storage media, but the real hardware
>> does have AHCI sata. I added sysbus-ahci at the right place but turns
>> out the SATA controller has some custom power/clock (not really
>> sure??) registers specific to this SoC. It sets/clears bits then polls
>> them back expecting them to change to the other value asynchronously.
>> The kernel device probe then times-out. So I subclassed sysbus-ahci
>> and added the missing registers and forced the polled registers to the
>> "I'm done" state. It works.
>
> Cool, are you going to submit patches for this?
>
>> I am using meta-sunxi Yocto-layer to build out the allwinner custom
>> kernel/rootfs etc, and with the clock and Sata changes I get a boot.
>> But when I change to the unedited kernel+dtb+rootfs I get stuck. RTC
>> messages are around the point of failure which is not modelled in
>> QEMU, so that is suspect.
>
> I don't know, this needs some investigation; on my side a recent
> multi_v7_defconfig kernel, unmodified sun4i-a10-cubieboard.dtb and a
> rootfs built with buildroot mounted through NFS work just fine, with
> the mentioned warnings regarding clk registers and also these:
>
What is your qemu command line ?

Thanks,
Guenter

> Ignoring attempt to switch CPSR_A flag from non-secure world with SCR.AW bit clear
> Ignoring attempt to switch CPSR_F flag from non-secure world with SCR.FW bit clear
>
> which probably would be solved by setting the property 'has_el3' of
> the CPU to false before realization.
>
> Beniamino
>

Re: [Qemu-devel] QEMU+Linux ARMv7A current state

[Beniamino Galvani]

On Sun, Oct 04, 2015 at 02:11:35PM -0700, Guenter Roeck wrote:
> What is your qemu command line ?

qemu-system-arm \
            -M cubieboard \
            -kernel ../linux/zImage-dtb \
            -serial stdio \
            -append "console=ttyS0 rw root=/dev/nfs nfsroot=10.0.0.1:/nfs,v3 ip=10.0.0.22" \
            -m 1024 \
            -net nic,vlan=0,model=allwinner-emac \
            -net tap,vlan=0,ifname=tap0,script=net-up.sh \
            -s \
            -d guest_errors

Beniamino

Re: [Qemu-devel] QEMU+Linux ARMv7A current state

[Peter Crosthwaite]

On Sun, Oct 4, 2015 at 12:56 PM, Beniamino Galvani <b.galvani@gmail.com> wrote:
> On Sat, Oct 03, 2015 at 02:31:08PM -0700, Peter Crosthwaite wrote:
>> QEMU cubieboard has no usable storage media, but the real hardware
>> does have AHCI sata. I added sysbus-ahci at the right place but turns
>> out the SATA controller has some custom power/clock (not really
>> sure??) registers specific to this SoC. It sets/clears bits then polls
>> them back expecting them to change to the other value asynchronously.
>> The kernel device probe then times-out. So I subclassed sysbus-ahci
>> and added the missing registers and forced the polled registers to the
>> "I'm done" state. It works.
>
> Cool, are you going to submit patches for this?
>
>> I am using meta-sunxi Yocto-layer to build out the allwinner custom
>> kernel/rootfs etc, and with the clock and Sata changes I get a boot.
>> But when I change to the unedited kernel+dtb+rootfs I get stuck. RTC
>> messages are around the point of failure which is not modelled in
>> QEMU, so that is suspect.
>
> I don't know, this needs some investigation; on my side a recent
> multi_v7_defconfig kernel, unmodified sun4i-a10-cubieboard.dtb and a
> rootfs built with buildroot mounted through NFS work just fine, with
> the mentioned warnings regarding clk registers and also these:
>

Looks like I am hanging in userspace, here is my hang:

[    4.064068] scsi 0:0:0:0: Direct-Access     ATA      QEMU HARDDISK
  50   PQ: 0 ANSI: 5
[    4.075977] sd 0:0:0:0: [sda] 16464 512-byte logical blocks: (8.42
MB/8.03 MiB)
[    4.082427] sd 0:0:0:0: [sda] Write Protect is off
[    4.085379] sd 0:0:0:0: [sda] Write cache: enabled, read cache:
enabled, doesn't support DPO or FUA
[    4.118143] sd 0:0:0:0: [sda] Attached SCSI disk
[    5.972385] sun4i-emac 1c0b000.ethernet eth0: Link is Up -
100Mbps/Full - flow control off
[    5.988629] Sending DHCP requests ., OK
[    6.011625] IP-Config: Got DHCP answer from 10.0.2.2, my address is 10.0.2.15
[    6.021258] IP-Config: Complete:
[    6.022075]      device=eth0, hwaddr=0a:80:64:4f:43:ec,
ipaddr=10.0.2.15, mask=255.255.255.0, gw=10.0.2.2
[    6.023220]      host=10.0.2.15, domain=, nis-domain=(none)
[    6.023751]      bootserver=10.0.2.2, rootserver=10.0.2.2, rootpath=
[    6.024436]      nameserver0=10.0.2.3
[    6.025489] usb2-vbus: disabling
[    6.025876] usb1-vbus: disabling
[    6.026194] vcc5v0: disabling
[    6.026514] vcc3v0: disabling
[    6.148152] EXT4-fs (sda): recovery complete
[    6.150995] EXT4-fs (sda): mounted filesystem with ordered data
mode. Opts: (null)
[    6.153751] VFS: Mounted root (ext4 filesystem) on device 8:0.
[    6.157800] devtmpfs: mounted
[    6.179159] Freeing unused kernel memory: 944K (c0d5c000 - c0e48000)
[    7.442057] udevd[76]: starting version 182
[   11.549686] random: nonblocking pool is initialized
[   13.536968] EXT4-fs (sda): re-mounted. Opts: data=ordered
[   16.163781] sunxi-rtc 1c20d00.rtc: Failed to set rtc time.

It is highly likely that someone from sunxi knows what is up, given
the yocto meta-layer works if you have anyone you can CC.

> Ignoring attempt to switch CPSR_A flag from non-secure world with SCR.AW bit clear
> Ignoring attempt to switch CPSR_F flag from non-secure world with SCR.FW bit clear
>
> which probably would be solved by setting the property 'has_el3' of
> the CPU to false before realization.
>

That sounds like a bug and should definately be fixed. We should have
cpus that do support EL3 saying they dont (due to legacy and lack of
testing) but not the other way round.

Regards,
Peter

> Beniamino

Re: [Qemu-devel] QEMU+Linux ARMv7A current state

[Peter Maydell]

On 4 October 2015 at 22:39, Peter Crosthwaite
<crosthwaitepeter@gmail.com> wrote:
> On Sun, Oct 4, 2015 at 12:56 PM, Beniamino Galvani <b.galvani@gmail.com> wrote:
>> Ignoring attempt to switch CPSR_A flag from non-secure world with SCR.AW bit clear
>> Ignoring attempt to switch CPSR_F flag from non-secure world with SCR.FW bit clear
>>
>> which probably would be solved by setting the property 'has_el3' of
>> the CPU to false before realization.
>>
>
> That sounds like a bug and should definately be fixed. We should have
> cpus that do support EL3 saying they dont (due to legacy and lack of
> testing) but not the other way round.

The Allwinner really does have EL3, so we are correct in not setting
has_el3 to false. We should figure out what's actually happening
here and fix whatever the underlying problem is. (Possibly the real
board firmware hands control of FIQ and Abort to an NS kernel by
setting SCR.AW/FW, which our built in loader doesn't do? Or the
guest really is buggy, or perhaps our GUEST_ERROR logging is a bit
overenthusiastic.)

thanks
-- PMM

Re: [Qemu-devel] QEMU+Linux ARMv7A current state

[Guenter Roeck]

On 10/04/2015 02:38 PM, Beniamino Galvani wrote:
> On Sun, Oct 04, 2015 at 02:11:35PM -0700, Guenter Roeck wrote:
>> What is your qemu command line ?
>
> qemu-system-arm \
>              -M cubieboard \
>              -kernel ../linux/zImage-dtb \
>              -serial stdio \
>              -append "console=ttyS0 rw root=/dev/nfs nfsroot=10.0.0.1:/nfs,v3 ip=10.0.0.22" \
>              -m 1024 \
>              -net nic,vlan=0,model=allwinner-emac \
>              -net tap,vlan=0,ifname=tap0,script=net-up.sh \
>              -s \
>              -d guest_errors
>
> Beniamino
>

With the mainline kernel and sun4i-a10-cubieboard.dtb, this gives me lots of

[    2.480983] Division by zero in kernel.
[    2.481074] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.3.0-rc3-00055-gdd36d7393d63 #1
[    2.481250] Hardware name: Allwinner A1X (Device Tree)
[    2.481376] [<c02186a8>] (unwind_backtrace) from [<c0213d84>] (show_stack+0x10/0x14)
[    2.481544] [<c0213d84>] (show_stack) from [<c047cff0>] (dump_stack+0x78/0x94)
[    2.481700] [<c047cff0>] (dump_stack) from [<c047bb38>] (Ldiv0+0x8/0x10)
[    2.481848] [<c047bb38>] (Ldiv0) from [<c0886ab0>] (sun4i_a10_get_mod0_factors+0x74/0xb8)
[    2.482024] [<c0886ab0>] (sun4i_a10_get_mod0_factors) from [<c08867e4>] (clk_factors_determine_rate+0x68/0xfc)
[    2.482231] [<c08867e4>] (clk_factors_determine_rate) from [<c0871070>] (clk_composite_determine_rate+0x94/0x1d0)
[    2.482443] [<c0871070>] (clk_composite_determine_rate) from [<c086c960>] (clk_core_round_rate_nolock+0x84/0xa8)
[    2.482654] [<c086c960>] (clk_core_round_rate_nolock) from [<c086dcbc>] (clk_round_rate+0x38/0x54)
[    2.482845] [<c086dcbc>] (clk_round_rate) from [<c084246c>] (sunxi_mmc_set_ios+0x9c/0x314)
[    2.483023] [<c084246c>] (sunxi_mmc_set_ios) from [<c0814564>] (mmc_power_up+0xf8/0x104)
[    2.483197] [<c0814564>] (mmc_power_up) from [<c08151d8>] (mmc_start_host+0x44/0x6c)
[    2.483363] [<c08151d8>] (mmc_start_host) from [<c0816244>] (mmc_add_host+0x58/0x7c)
[    2.483528] [<c0816244>] (mmc_add_host) from [<c0842b6c>] (sunxi_mmc_probe+0x488/0x590)
[    2.483701] [<c0842b6c>] (sunxi_mmc_probe) from [<c06287d0>] (platform_drv_probe+0x48/0xa4)

Do you have a special devicetree file ?

It also doesn't seem to accept the qemu "initrd" argument, which is unexpected.

Any idea what might be wrong ? Here is the command line I tried.

qemu-system-arm \
	-M cubieboard -kernel arch/arm/boot/zImage -no-reboot \
	-initrd core-image-minimal-qemuarm.cpio \
	-append "rdinit=/sbin/init console=ttyS0" \
	-nographic -monitor none -serial stdio \
	-dtb arch/arm/boot/dts/sun4i-a10-cubieboard.dtb

Thanks,
Guenter

Re: [Qemu-devel] QEMU+Linux ARMv7A current state

[Peter Crosthwaite]

On Sun, Oct 4, 2015 at 6:08 PM, Guenter Roeck <linux@roeck-us.net> wrote:
> On 10/04/2015 02:38 PM, Beniamino Galvani wrote:
>>
>> On Sun, Oct 04, 2015 at 02:11:35PM -0700, Guenter Roeck wrote:
>>>
>>> What is your qemu command line ?
>>
>>
>> qemu-system-arm \
>>              -M cubieboard \
>>              -kernel ../linux/zImage-dtb \
>>              -serial stdio \
>>              -append "console=ttyS0 rw root=/dev/nfs
>> nfsroot=10.0.0.1:/nfs,v3 ip=10.0.0.22" \
>>              -m 1024 \
>>              -net nic,vlan=0,model=allwinner-emac \
>>              -net tap,vlan=0,ifname=tap0,script=net-up.sh \
>>              -s \
>>              -d guest_errors
>>
>> Beniamino
>>
>
> With the mainline kernel and sun4i-a10-cubieboard.dtb, this gives me lots of
>
> [    2.480983] Division by zero in kernel.
> [    2.481074] CPU: 0 PID: 1 Comm: swapper/0 Not tainted
> 4.3.0-rc3-00055-gdd36d7393d63 #1
> [    2.481250] Hardware name: Allwinner A1X (Device Tree)
> [    2.481376] [<c02186a8>] (unwind_backtrace) from [<c0213d84>]
> (show_stack+0x10/0x14)
> [    2.481544] [<c0213d84>] (show_stack) from [<c047cff0>]
> (dump_stack+0x78/0x94)
> [    2.481700] [<c047cff0>] (dump_stack) from [<c047bb38>] (Ldiv0+0x8/0x10)
> [    2.481848] [<c047bb38>] (Ldiv0) from [<c0886ab0>]
> (sun4i_a10_get_mod0_factors+0x74/0xb8)
> [    2.482024] [<c0886ab0>] (sun4i_a10_get_mod0_factors) from [<c08867e4>]
> (clk_factors_determine_rate+0x68/0xfc)
> [    2.482231] [<c08867e4>] (clk_factors_determine_rate) from [<c0871070>]
> (clk_composite_determine_rate+0x94/0x1d0)
> [    2.482443] [<c0871070>] (clk_composite_determine_rate) from [<c086c960>]
> (clk_core_round_rate_nolock+0x84/0xa8)
> [    2.482654] [<c086c960>] (clk_core_round_rate_nolock) from [<c086dcbc>]
> (clk_round_rate+0x38/0x54)
> [    2.482845] [<c086dcbc>] (clk_round_rate) from [<c084246c>]
> (sunxi_mmc_set_ios+0x9c/0x314)
> [    2.483023] [<c084246c>] (sunxi_mmc_set_ios) from [<c0814564>]
> (mmc_power_up+0xf8/0x104)
> [    2.483197] [<c0814564>] (mmc_power_up) from [<c08151d8>]
> (mmc_start_host+0x44/0x6c)
> [    2.483363] [<c08151d8>] (mmc_start_host) from [<c0816244>]
> (mmc_add_host+0x58/0x7c)
> [    2.483528] [<c0816244>] (mmc_add_host) from [<c0842b6c>]
> (sunxi_mmc_probe+0x488/0x590)
> [    2.483701] [<c0842b6c>] (sunxi_mmc_probe) from [<c06287d0>]
> (platform_drv_probe+0x48/0xa4)
>

I think that's the expected warnings we have been ignoring. I have a
hack to make them go away.

> Do you have a special devicetree file ?
>
> It also doesn't seem to accept the qemu "initrd" argument, which is
> unexpected.
>

Yes I noticed the same and went to the SATA solution.

Regards,
Peter

> Any idea what might be wrong ? Here is the command line I tried.
>
> qemu-system-arm \
>         -M cubieboard -kernel arch/arm/boot/zImage -no-reboot \
>         -initrd core-image-minimal-qemuarm.cpio \
>         -append "rdinit=/sbin/init console=ttyS0" \
>         -nographic -monitor none -serial stdio \
>         -dtb arch/arm/boot/dts/sun4i-a10-cubieboard.dtb
>
> Thanks,
> Guenter
>

Re: [Qemu-devel] QEMU+Linux ARMv7A current state

[Guenter Roeck]

On 10/04/2015 07:21 PM, Peter Crosthwaite wrote:
> On Sun, Oct 4, 2015 at 6:08 PM, Guenter Roeck <linux@roeck-us.net> wrote:
>> On 10/04/2015 02:38 PM, Beniamino Galvani wrote:
>>>
>>> On Sun, Oct 04, 2015 at 02:11:35PM -0700, Guenter Roeck wrote:
>>>>
>>>> What is your qemu command line ?
>>>
>>>
>>> qemu-system-arm \
>>>               -M cubieboard \
>>>               -kernel ../linux/zImage-dtb \
>>>               -serial stdio \
>>>               -append "console=ttyS0 rw root=/dev/nfs
>>> nfsroot=10.0.0.1:/nfs,v3 ip=10.0.0.22" \
>>>               -m 1024 \
>>>               -net nic,vlan=0,model=allwinner-emac \
>>>               -net tap,vlan=0,ifname=tap0,script=net-up.sh \
>>>               -s \
>>>               -d guest_errors
>>>
>>> Beniamino
>>>
>>
>> With the mainline kernel and sun4i-a10-cubieboard.dtb, this gives me lots of
>>
>> [    2.480983] Division by zero in kernel.
>> [    2.481074] CPU: 0 PID: 1 Comm: swapper/0 Not tainted
>> 4.3.0-rc3-00055-gdd36d7393d63 #1
>> [    2.481250] Hardware name: Allwinner A1X (Device Tree)
>> [    2.481376] [<c02186a8>] (unwind_backtrace) from [<c0213d84>]
>> (show_stack+0x10/0x14)
>> [    2.481544] [<c0213d84>] (show_stack) from [<c047cff0>]
>> (dump_stack+0x78/0x94)
>> [    2.481700] [<c047cff0>] (dump_stack) from [<c047bb38>] (Ldiv0+0x8/0x10)
>> [    2.481848] [<c047bb38>] (Ldiv0) from [<c0886ab0>]
>> (sun4i_a10_get_mod0_factors+0x74/0xb8)
>> [    2.482024] [<c0886ab0>] (sun4i_a10_get_mod0_factors) from [<c08867e4>]
>> (clk_factors_determine_rate+0x68/0xfc)
>> [    2.482231] [<c08867e4>] (clk_factors_determine_rate) from [<c0871070>]
>> (clk_composite_determine_rate+0x94/0x1d0)
>> [    2.482443] [<c0871070>] (clk_composite_determine_rate) from [<c086c960>]
>> (clk_core_round_rate_nolock+0x84/0xa8)
>> [    2.482654] [<c086c960>] (clk_core_round_rate_nolock) from [<c086dcbc>]
>> (clk_round_rate+0x38/0x54)
>> [    2.482845] [<c086dcbc>] (clk_round_rate) from [<c084246c>]
>> (sunxi_mmc_set_ios+0x9c/0x314)
>> [    2.483023] [<c084246c>] (sunxi_mmc_set_ios) from [<c0814564>]
>> (mmc_power_up+0xf8/0x104)
>> [    2.483197] [<c0814564>] (mmc_power_up) from [<c08151d8>]
>> (mmc_start_host+0x44/0x6c)
>> [    2.483363] [<c08151d8>] (mmc_start_host) from [<c0816244>]
>> (mmc_add_host+0x58/0x7c)
>> [    2.483528] [<c0816244>] (mmc_add_host) from [<c0842b6c>]
>> (sunxi_mmc_probe+0x488/0x590)
>> [    2.483701] [<c0842b6c>] (sunxi_mmc_probe) from [<c06287d0>]
>> (platform_drv_probe+0x48/0xa4)
>>
>
> I think that's the expected warnings we have been ignoring. I have a
> hack to make them go away.
>

Division by zero isn't something that should be ignored.
Any idea where it is coming from ?

Also, is your hack in the kernel or in qemu ?

>> Do you have a special devicetree file ?
>>
>> It also doesn't seem to accept the qemu "initrd" argument, which is
>> unexpected.
>>
>
> Yes I noticed the same and went to the SATA solution.
>

Here is a one-line qemu fix for the initrd problem.

diff --git a/hw/arm/cubieboard.c b/hw/arm/cubieboard.c
index 1582250..db3ec40 100644
--- a/hw/arm/cubieboard.c
+++ b/hw/arm/cubieboard.c
@@ -71,6 +71,7 @@ static void cubieboard_init(MachineState *machine)
      cubieboard_binfo.ram_size = machine->ram_size;
      cubieboard_binfo.kernel_filename = machine->kernel_filename;
      cubieboard_binfo.kernel_cmdline = machine->kernel_cmdline;
+    cubieboard_binfo.initrd_filename = machine->initrd_filename;
      arm_load_kernel(&s->a10->cpu, &cubieboard_binfo);
  }

Guess that is less complex than getting sata to work ?

Thanks,
Guenter

Re: [Qemu-devel] QEMU+Linux ARMv7A current state

[John Snow]

On 10/03/2015 05:31 PM, Peter Crosthwaite wrote:
> Hi,
> 
> I have done an audit of the ARMv7 boards to see what can boot a
> vanilla linux kernel. The basic approach is to build ARM
> multi_v7_defconfig kernel and boot QEMU using the DTBs built out by
> the kernel. The intersection of what mainline Linux has a DTB for and
> what QEMU models is tested. The boards that do/should work include:
> 
>     vexpress-a9
>     vexpress-a15
>     smdkc210 (exynos)
>     xilinx-zynq-a9
>     cubieboard (allwinner)
>     highbank
>     midway
>     virt
> 
> Have I missed anything?
> 
> The basic rule is, no source code, config, or DTB edits allowed (i.e.
> what is a day0 user of QEMU+Linux going to experience?).
> 
> So going board-by-board, these are the issues.
> 
> highbank/midway:
> 
> The kernel expects a secure monitor to be present which does at-least
> two things:
> 
> * PSCI
> * Some cache-maintainance ops.
> 
> QEMU has PSCI support, but the 0.1 command codings specified in the
> DTB are different to QEMUs 0.1 encodings. So I switched monitor mode
> for the CPUs on (it was conservatively set to off by the original
> authors of the QEMU ARM monitor support) and turned PSCI on. I had to
> write some patches to remap the PSCI encodings to highbank's custom
> ones and then the kernel at least entry-points all 4 cpus. It then
> deadlocks on something that I am yet to figure out. So the PSCI SMP
> stuff is shelved.
> 
> The cache-maintenance is a bigger issue, that stops even a single core
> boot. The kernel cache controller driver is doing an smc that QEMU
> goes haywire on (as there simply is no firmware to catch the smc). But
> caches in QEMU are a nop, so I wrote some monitor firmware that just
> erets the smc without action and it works.
> 
> I am using Sata (sysbus-ahci) for the boot media which works straight
> out of the box.
> 
> So single core highbank works with the dummy monitor firmware and the
> monitor mode restrictor removed.
> 
> cubieboard:
> 
> The kernel expects a system level clock controller device that isn't
> modelled. A dummy model which hardcodes all registers to a know reset
> value and just lets the kernel read and write whatever it wants
> resolves. I'm not 100% sure whether this is needed but the kernel
> prints multiple errors with many pages of traces without. So it is at
> least needed to reduce the dmesg noise.
> 
> QEMU cubieboard has no usable storage media, but the real hardware
> does have AHCI sata. I added sysbus-ahci at the right place but turns
> out the SATA controller has some custom power/clock (not really
> sure??) registers specific to this SoC. It sets/clears bits then polls
> them back expecting them to change to the other value asynchronously.
> The kernel device probe then times-out. So I subclassed sysbus-ahci
> and added the missing registers and forced the polled registers to the
> "I'm done" state. It works.
> 

I'm looking into the cubieboard now. Is our emulation based on any
particular model? (1-4?)

I'm trying to see if I can find anything that resembles a spec to see
what kind of registers this SoC has for its SATA controller.

Feel free to point me towards your patches, too.

> I am using meta-sunxi Yocto-layer to build out the allwinner custom
> kernel/rootfs etc, and with the clock and Sata changes I get a boot.
> But when I change to the unedited kernel+dtb+rootfs I get stuck. RTC
> messages are around the point of failure which is not modelled in
> QEMU, so that is suspect.
> 
> So with a dummy clock-controller and a slightly modified Sata
> controller (as a new device) cubieboard progresses, but it is still
> stuck on something that looks RTC related.
> 
> xilinx-zynq-a9:
> 
> The kernel wants to use cpufreq to set the CPU frequency. The kernel
> assumes ownership of the CPU clock divider but not the PLLs. QEMU uses
> the HW default for the silicon (x26) for the PLL which is incompatible
> with the dtb-set CPU frequency. The kernel boot asserts this and
> crashes. The ideal solution is to provide some sort of pre-boot
> firmware to set it to a compatible value. In real HW, this is the FSBL
> bootloader.
> 
> TBH I think this is a kernel bug, as since cpufreq can be deconfigured
> it is ultimately optional. This means it shouldn't be a showstopper on
> a boot. The kernel should just switch off cpufreq and proceed and
> behave as if the feature was never configured in the kernel.
> 
> There is also a critical bug in the SLCR model (my bad) where write to
> registers are ignored. I have a patch.
> 
> More reading about the SLCR issues here (thanks to Guenter for getting
> the ball rolling here):
> 
> https://lists.gnu.org/archive/html/qemu-devel/2015-09/msg03374.html
> 
> I have a patch with sets up the needed SLCR values from software but
> it needs more work.
> 
> The kernel also expect the ADC device which is absent. Thanks to
> Guenter who patched this in which is under review:
> 
> https://lists.gnu.org/archive/html/qemu-devel/2015-09/msg03375.html
> 
> I am using SD card for the rootfs media. SD has a bug where it cannot
> handle a backing file of a non-round size. I am just passing the ext3
> image in unchanged (so unpadded) to QEMU and QEMU rounds down the
> image to the SD block size (512k!!), so I am losing us to 512k off the
> end of my ext3 filesystem. I have a hacky patch that changes this to
> round-up.
> 
> With the minimal firmware, slcr bugfix, Guenters ADC, the SD hack a
> vanilla Zynq will be in good shape.
> 
> virt:
> 
> Virt largely works, but there are no immediately obvious
> storage/network options that are supported by the kernel. So I have to
> make some exceptions to the kernel config rule, that is, I add:
> 
> CONFIG_VIRTIO_PCI=y"
> CONFIG_VIRTIO_BLK=y"
> CONFIG_VIRTIO_NET=y"
> 
> To get disks and network for the virt board.
> 
> Could we add these configs to mainline Linux's defconfig? Note, we
> don't need to add a DTS for this board, as QEMU's bootloader will
> generate and provide it for us.
> 
> There is the known highmem PCI issue that is currently under discussion:
> 
> https://lists.gnu.org/archive/html/qemu-devel/2015-09/msg06529.html
> 
> So aside from the missing VIRTIO kernel configs and highmem bug, virt
> is in good shape.
> 
> vexpress:
> 
> vexpress boots up to rootfs probing, however the only storage media
> that seems to be supported is flash, which doesn't have the needed
> kernel configs to boot from (not 100% sure on that but I gave up
> quickly). What are people who use this board in real hardware doing?
> It may just work best with initrd style boots.
> 
> exynos:
> 
> QEMU models an Exynos skdkc210 but there is only a 310 available in
> QEMU dtbs. QEMU exynos however does not model a lot on the board level
> so we can largely get away with booting 310 dtb on 210.
> 
> Similar to vexpress, the exynos board has a lack of usable boot media.
> The board does model USB (EHCI), however the DTB disables it as the
> smdk310 board does not support it. So the alternative would be to use
> one of the more full-featured exynos DTBs  (with way more peripherals)
> from mainline Linux, but the kernel is liable to then probe all sorts
> of missing hardware.
> 
> Exynos SDHCI support was on list for a few revs and quite some time,
> merging it and using SD may solve the problem.
> 
> I have not attempted the SMP support.
> 
> Unlike all other boards, Exynos is quite consuming of RAM. All other
> boards work with Yocto's default of 128MB but Exynos crashes unless
> you jack it up. So users need to be careful of the -m option.
> 
> realview:
> 
> Realview is not tested, but I discuss it here as it is a bit of a red
> herring. The kernel has dtb support for the realview-1176 board,
> however this is not modelled by QEMU. I creating a Realview variant
> with 1176 in QEMU, however the memory maps of all the peripherals were
> mismatched.
> 
> Turns, out, QEMU is is modelling the realview FPGA emulation platform
> which has a different peripheral map to the 1176 realview board
> proper. So QEMU doesn't support the realview boards, just the FPGA
> emulation platform, for which I can't find a kernel dtb. Should we
> upstream one with some dtsi's for the CPU variation supported in QEMU?
> 
> So Realview does not work within my testing parameters with no
> immediate prospects.
> 
> how I am testing:
> 
> So this is all powered by the Yocto project (Poky). I got some good
> help from Nathan Rossi. I have a poky fork which changes the qemuarm
> target to build my mainline (4.2.1) kernel (multi_v7_defconfig +
> VIRTIO), all the DTBs and a usable rootfs. You can then specify the
> type of machine (from the list above) to yocto's runqemu command. The
> command sets up boot media and network automatically.
> 
> To use it:
> 
> git clone https://github.com/pcrost/poky.git
> cd poky
> source ./oe-init-build-env
> MACHINE=qemuarm bitbake core-image-minimal #this takes a while
> QEMUBIN=/path/to/qemu-arm MACHINE_SUBTYPE=virt runqemu qemuarm slirp
> 
> The QEMUBIN is optional as Yocto does build QEMU for you, but this
> lets you BYO QEMU if you are doing qemu development outside of the
> Yocto flow. Change MACHINE_SUBTYPE to one of the supported boards to
> see results. I suggest starting with virt. Skip cubieboard, that
> assumes my SATA patches. Everything else you will see varying degrees
> of success. Pass qemuparams="-m 256" to runqemu for the smdkc200
> (Exynos) board. Highbank and Midway will blank screen as the
> outstanding issues happen pre-UART. A meaningful logbuf can be pulled
> from RAM over the monitor. Useful instructions here:
> 
> http://www.wiki.xilinx.com/QEMU-Linux+Kernel+logbuf+Extraction
> 
> Some QEMU patches to follow to fix a few of the things mentioned.
> 
> We should get this into regression testing. Yocto has some automated
> testing features in it's own right that we should be able to leverage.
> Yet to investigate (Richard pasted some stuff on an earlier thread).
> 
> The exercise should be redone for ARM64, ARMv5 and ARMv6 and ARMv7M,
> then other arches (many of which Poky supports).
> 
> Regards,
> Peter
> 

Re: [Qemu-devel] QEMU+Linux ARMv7A current state

[Beniamino Galvani]

On Mon, Oct 05, 2015 at 11:13:33AM -0400, John Snow wrote:
> I'm looking into the cubieboard now. Is our emulation based on any
> particular model? (1-4?)

The first model, the one with Allwinner A10.

> I'm trying to see if I can find anything that resembles a spec to see
> what kind of registers this SoC has for its SATA controller.

There is some documentation on the SoC here, but apparently nothing on
SATA:

http://dl.linux-sunxi.org/A10/

Beniamino

Re: [Qemu-devel] QEMU+Linux ARMv7A current state

[John Snow]

On 10/05/2015 04:44 PM, Beniamino Galvani wrote:
> On Mon, Oct 05, 2015 at 11:13:33AM -0400, John Snow wrote:
>> I'm looking into the cubieboard now. Is our emulation based on any
>> particular model? (1-4?)
> 
> The first model, the one with Allwinner A10.
> 
>> I'm trying to see if I can find anything that resembles a spec to see
>> what kind of registers this SoC has for its SATA controller.
> 
> There is some documentation on the SoC here, but apparently nothing on
> SATA:
> 
> http://dl.linux-sunxi.org/A10/
> 
> Beniamino
> 

http://linux-sunxi.org/SATA looks relevant...

http://dl.cubieforums.com/files/pdf/A10_development_board_user_manual--2011.9.23_English.pdf
mentions the feature list:

The board provides one SATA interface, which features:
- Support SATA 1.5Gb/s, and SATA 3.0Gb/s
- Compliant to SATA Spec. 2.6, and AHCI Revision 1.3 Specifications
- Support industry-standard AMBA High-Performance Bus (AHB) and fully
compliant to the AMBA Specification, Revision 2.0; Support 32-bit Little
Endian
- OOB signaling detection and generation
- SATA 1.5Gb/s and SATA 3.0Gb/s speed negotiation when Tx OOB signaling
is selected
- Support device hot-plugging
- Support power management features including automatic Partial to
Slumber transition
- Internal DMA Engine for Command and Data Transactions
- Support hardware-assisted Native Command Queuing (NCQ) for up to
32-entries
- Support external SATA (eSATA)

I can't find anything else, though. I'll just have to wait from Peter to
see what registers he needed to modify to get it working.

Thanks,
--js

Re: [Qemu-devel] QEMU+Linux ARMv7A current state

[Peter Crosthwaite]

On Sun, Oct 4, 2015 at 9:09 PM, Guenter Roeck <linux@roeck-us.net> wrote:
> On 10/04/2015 07:21 PM, Peter Crosthwaite wrote:
>>
>> On Sun, Oct 4, 2015 at 6:08 PM, Guenter Roeck <linux@roeck-us.net> wrote:
>>>
>>> On 10/04/2015 02:38 PM, Beniamino Galvani wrote:
>>>>
>>>>
>>>> On Sun, Oct 04, 2015 at 02:11:35PM -0700, Guenter Roeck wrote:
>>>>>
>>>>>
>>>>> What is your qemu command line ?
>>>>
>>>>
>>>>
>>>> qemu-system-arm \
>>>>               -M cubieboard \
>>>>               -kernel ../linux/zImage-dtb \
>>>>               -serial stdio \
>>>>               -append "console=ttyS0 rw root=/dev/nfs
>>>> nfsroot=10.0.0.1:/nfs,v3 ip=10.0.0.22" \
>>>>               -m 1024 \
>>>>               -net nic,vlan=0,model=allwinner-emac \
>>>>               -net tap,vlan=0,ifname=tap0,script=net-up.sh \
>>>>               -s \
>>>>               -d guest_errors
>>>>
>>>> Beniamino
>>>>
>>>
>>> With the mainline kernel and sun4i-a10-cubieboard.dtb, this gives me lots
>>> of
>>>
>>> [    2.480983] Division by zero in kernel.
>>> [    2.481074] CPU: 0 PID: 1 Comm: swapper/0 Not tainted
>>> 4.3.0-rc3-00055-gdd36d7393d63 #1
>>> [    2.481250] Hardware name: Allwinner A1X (Device Tree)
>>> [    2.481376] [<c02186a8>] (unwind_backtrace) from [<c0213d84>]
>>> (show_stack+0x10/0x14)
>>> [    2.481544] [<c0213d84>] (show_stack) from [<c047cff0>]
>>> (dump_stack+0x78/0x94)
>>> [    2.481700] [<c047cff0>] (dump_stack) from [<c047bb38>]
>>> (Ldiv0+0x8/0x10)
>>> [    2.481848] [<c047bb38>] (Ldiv0) from [<c0886ab0>]
>>> (sun4i_a10_get_mod0_factors+0x74/0xb8)
>>> [    2.482024] [<c0886ab0>] (sun4i_a10_get_mod0_factors) from
>>> [<c08867e4>]
>>> (clk_factors_determine_rate+0x68/0xfc)
>>> [    2.482231] [<c08867e4>] (clk_factors_determine_rate) from
>>> [<c0871070>]
>>> (clk_composite_determine_rate+0x94/0x1d0)
>>> [    2.482443] [<c0871070>] (clk_composite_determine_rate) from
>>> [<c086c960>]
>>> (clk_core_round_rate_nolock+0x84/0xa8)
>>> [    2.482654] [<c086c960>] (clk_core_round_rate_nolock) from
>>> [<c086dcbc>]
>>> (clk_round_rate+0x38/0x54)
>>> [    2.482845] [<c086dcbc>] (clk_round_rate) from [<c084246c>]
>>> (sunxi_mmc_set_ios+0x9c/0x314)
>>> [    2.483023] [<c084246c>] (sunxi_mmc_set_ios) from [<c0814564>]
>>> (mmc_power_up+0xf8/0x104)
>>> [    2.483197] [<c0814564>] (mmc_power_up) from [<c08151d8>]
>>> (mmc_start_host+0x44/0x6c)
>>> [    2.483363] [<c08151d8>] (mmc_start_host) from [<c0816244>]
>>> (mmc_add_host+0x58/0x7c)
>>> [    2.483528] [<c0816244>] (mmc_add_host) from [<c0842b6c>]
>>> (sunxi_mmc_probe+0x488/0x590)
>>> [    2.483701] [<c0842b6c>] (sunxi_mmc_probe) from [<c06287d0>]
>>> (platform_drv_probe+0x48/0xa4)
>>>
>>
>> I think that's the expected warnings we have been ignoring. I have a
>> hack to make them go away.
>>
>
> Division by zero isn't something that should be ignored.
> Any idea where it is coming from ?
>
> Also, is your hack in the kernel or in qemu ?
>
>>> Do you have a special devicetree file ?
>>>
>>> It also doesn't seem to accept the qemu "initrd" argument, which is
>>> unexpected.
>>>
>>
>> Yes I noticed the same and went to the SATA solution.
>>
>
> Here is a one-line qemu fix for the initrd problem.
>
> diff --git a/hw/arm/cubieboard.c b/hw/arm/cubieboard.c
> index 1582250..db3ec40 100644
> --- a/hw/arm/cubieboard.c
> +++ b/hw/arm/cubieboard.c
> @@ -71,6 +71,7 @@ static void cubieboard_init(MachineState *machine)
>      cubieboard_binfo.ram_size = machine->ram_size;
>      cubieboard_binfo.kernel_filename = machine->kernel_filename;
>      cubieboard_binfo.kernel_cmdline = machine->kernel_cmdline;
> +    cubieboard_binfo.initrd_filename = machine->initrd_filename;
>      arm_load_kernel(&s->a10->cpu, &cubieboard_binfo);
>  }
>
> Guess that is less complex than getting sata to work ?
>

Makes sense. Do you (or Beniamino) want to spin the patch? Ideally we
should support both SATA and initrd.

Regards,
Peter

> Thanks,
> Guenter
>

Re: [Qemu-devel] QEMU+Linux ARMv7A current state

[Peter Crosthwaite]

On Sun, Oct 4, 2015 at 12:56 PM, Beniamino Galvani <b.galvani@gmail.com> wrote:
> On Sat, Oct 03, 2015 at 02:31:08PM -0700, Peter Crosthwaite wrote:
>> QEMU cubieboard has no usable storage media, but the real hardware
>> does have AHCI sata. I added sysbus-ahci at the right place but turns
>> out the SATA controller has some custom power/clock (not really
>> sure??) registers specific to this SoC. It sets/clears bits then polls
>> them back expecting them to change to the other value asynchronously.
>> The kernel device probe then times-out. So I subclassed sysbus-ahci
>> and added the missing registers and forced the polled registers to the
>> "I'm done" state. It works.
>
> Cool, are you going to submit patches for this?
>
>> I am using meta-sunxi Yocto-layer to build out the allwinner custom
>> kernel/rootfs etc, and with the clock and Sata changes I get a boot.
>> But when I change to the unedited kernel+dtb+rootfs I get stuck. RTC
>> messages are around the point of failure which is not modelled in
>> QEMU, so that is suspect.
>
> I don't know, this needs some investigation; on my side a recent
> multi_v7_defconfig kernel, unmodified sun4i-a10-cubieboard.dtb and a
> rootfs built with buildroot mounted through NFS work just fine, with
> the mentioned warnings regarding clk registers and also these:
>

False alarm. There were some guest-side issues getting the login
console. With just the SATA (and/or Guenters initrd fix) we are good.

I'm dropping the CCM patches as they don't add anything real and don't
stop any real functionality.

Regards,
Peter

> Ignoring attempt to switch CPSR_A flag from non-secure world with SCR.AW bit clear
> Ignoring attempt to switch CPSR_F flag from non-secure world with SCR.FW bit clear
>
> which probably would be solved by setting the property 'has_el3' of
> the CPU to false before realization.
>
> Beniamino