KVM on ARM64

KVM on ARM64

[Mathew Li]

Hi,

I have a quick question. How do we add a hard disk to the qemu ARM VM?

I tried:

qemu-system-aarch64 -machine virt -hda disk.img -kernel image -initrd initrd.img

qemu-system-aarch64 -machine virt -sd disk.img -kernel image -initrd initrd.img

qemu-system-aarch64 -machine virt -mtdblock disk.img -kernel image
-initrd initrd.img

Nothing seems to work. I am not able to see any disk (i.e. dev/sdX)
inside guest OS.

Can someone please guide on this?

Many thanks in advance!
Mathew

Re: KVM on ARM64

[Wei Huang]

On 08/04/2014 07:35 PM, Mathew Li wrote:
> Hi,
>
> I have a quick question. How do we add a hard disk to the qemu ARM VM?
>
> I tried:
>
> qemu-system-aarch64 -machine virt -hda disk.img -kernel image -initrd initrd.img
>
> qemu-system-aarch64 -machine virt -sd disk.img -kernel image -initrd initrd.img
>
> qemu-system-aarch64 -machine virt -mtdblock disk.img -kernel image
> -initrd initrd.img
>
> Nothing seems to work. I am not able to see any disk (i.e. dev/sdX)
> inside guest OS.
>
You can expose disk to guest VM via virtio, then guest can mount 
/dev/vda as rootfs.

-Wei

> Can someone please guide on this?
>
> Many thanks in advance!
> Mathew
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Re: KVM on ARM64

[Joel Schopp]

On 08/04/2014 07:35 PM, Mathew Li wrote:
> Hi,
>
> I have a quick question. How do we add a hard disk to the qemu ARM VM?
>
> I tried:
>
> qemu-system-aarch64 -machine virt -hda disk.img -kernel image -initrd initrd.img
>
> qemu-system-aarch64 -machine virt -sd disk.img -kernel image -initrd initrd.img
>
> qemu-system-aarch64 -machine virt -mtdblock disk.img -kernel image
> -initrd initrd.img
>
> Nothing seems to work. I am not able to see any disk (i.e. dev/sdX)
> inside guest OS.
I've been running something like this:

qemu-system-aarch64 -smp 1 --enable-kvm -nographic -netdev tap,id=t0,ifname=tap0,script=no,downscript=no,vhost=on -device virtio-net-device,netdev=t0,id=nic0 \
-kernel /extra/rootfs/boot/Image -drive file=/extra/rootfs.img,id=fs -device virtio-blk-device,drive=fs -m 512 -M virt -cpu host -append "console=ttyAMA0 console=ttyS0 root=/dev/vda"


On my system -smp 2 or higher hangs in the guest kernel.

Re: KVM on ARM64

[Christoffer Dall]

On Tue, Aug 5, 2014 at 4:18 PM, Joel Schopp <joel.schopp@amd.com> wrote:
>
> On 08/04/2014 07:35 PM, Mathew Li wrote:
>> Hi,
>>
>> I have a quick question. How do we add a hard disk to the qemu ARM VM?
>>
>> I tried:
>>
>> qemu-system-aarch64 -machine virt -hda disk.img -kernel image -initrd initrd.img
>>
>> qemu-system-aarch64 -machine virt -sd disk.img -kernel image -initrd initrd.img
>>
>> qemu-system-aarch64 -machine virt -mtdblock disk.img -kernel image
>> -initrd initrd.img
>>
>> Nothing seems to work. I am not able to see any disk (i.e. dev/sdX)
>> inside guest OS.
> I've been running something like this:
>
> qemu-system-aarch64 -smp 1 --enable-kvm -nographic -netdev tap,id=t0,ifname=tap0,script=no,downscript=no,vhost=on -device virtio-net-device,netdev=t0,id=nic0 \
> -kernel /extra/rootfs/boot/Image -drive file=/extra/rootfs.img,id=fs -device virtio-blk-device,drive=fs -m 512 -M virt -cpu host -append "console=ttyAMA0 console=ttyS0 root=/dev/vda"
>
>
> On my system -smp 2 or higher hangs in the guest kernel.

The -smp 2 hang issue is probably due to a missing PSCI v0.2 follow-up
patch to QEMU, you can try:
https://git.linaro.org/people/christoffer.dall/qemu-arm.git/shortlog/refs/heads/psci2-smp-fix

[disclaimer: there may be a better fix somewhere on the qemu list, I
haven't kept track the last couple of days]

-Christoffer

Re: KVM on ARM64

[Joel Schopp]

It turns out that after a recent rebase of my kernel and qemu to the
latest the problem is fixed.  Rather than hunt down what fixed it I'm
just accepting the win and moving on. -smp 4 now works.

-Joel

On 08/06/2014 11:15 AM, Christoffer Dall wrote:
> On Tue, Aug 5, 2014 at 4:18 PM, Joel Schopp <joel.schopp@amd.com> wrote:
>> On 08/04/2014 07:35 PM, Mathew Li wrote:
>>> Hi,
>>>
>>> I have a quick question. How do we add a hard disk to the qemu ARM VM?
>>>
>>> I tried:
>>>
>>> qemu-system-aarch64 -machine virt -hda disk.img -kernel image -initrd initrd.img
>>>
>>> qemu-system-aarch64 -machine virt -sd disk.img -kernel image -initrd initrd.img
>>>
>>> qemu-system-aarch64 -machine virt -mtdblock disk.img -kernel image
>>> -initrd initrd.img
>>>
>>> Nothing seems to work. I am not able to see any disk (i.e. dev/sdX)
>>> inside guest OS.
>> I've been running something like this:
>>
>> qemu-system-aarch64 -smp 1 --enable-kvm -nographic -netdev tap,id=t0,ifname=tap0,script=no,downscript=no,vhost=on -device virtio-net-device,netdev=t0,id=nic0 \
>> -kernel /extra/rootfs/boot/Image -drive file=/extra/rootfs.img,id=fs -device virtio-blk-device,drive=fs -m 512 -M virt -cpu host -append "console=ttyAMA0 console=ttyS0 root=/dev/vda"
>>
>>
>> On my system -smp 2 or higher hangs in the guest kernel.
> The -smp 2 hang issue is probably due to a missing PSCI v0.2 follow-up
> patch to QEMU, you can try:
> https://git.linaro.org/people/christoffer.dall/qemu-arm.git/shortlog/refs/heads/psci2-smp-fix
>
> [disclaimer: there may be a better fix somewhere on the qemu list, I
> haven't kept track the last couple of days]
>
> -Christoffer

Re: KVM on ARM64

[Mathew Li]

Great. VirtIO works for me. Thanks for your help folks!

Is there is any other way to add virtual disk, more like a traditional
disk to qemu-system-aarch64? For example IDE disk or SATA disk or
maybe as a SCSI disk?

On Wed, Aug 6, 2014 at 9:48 AM, Joel Schopp <joel.schopp@amd.com> wrote:
> It turns out that after a recent rebase of my kernel and qemu to the
> latest the problem is fixed.  Rather than hunt down what fixed it I'm
> just accepting the win and moving on. -smp 4 now works.
>
> -Joel
>
> On 08/06/2014 11:15 AM, Christoffer Dall wrote:
>> On Tue, Aug 5, 2014 at 4:18 PM, Joel Schopp <joel.schopp@amd.com> wrote:
>>> On 08/04/2014 07:35 PM, Mathew Li wrote:
>>>> Hi,
>>>>
>>>> I have a quick question. How do we add a hard disk to the qemu ARM VM?
>>>>
>>>> I tried:
>>>>
>>>> qemu-system-aarch64 -machine virt -hda disk.img -kernel image -initrd initrd.img
>>>>
>>>> qemu-system-aarch64 -machine virt -sd disk.img -kernel image -initrd initrd.img
>>>>
>>>> qemu-system-aarch64 -machine virt -mtdblock disk.img -kernel image
>>>> -initrd initrd.img
>>>>
>>>> Nothing seems to work. I am not able to see any disk (i.e. dev/sdX)
>>>> inside guest OS.
>>> I've been running something like this:
>>>
>>> qemu-system-aarch64 -smp 1 --enable-kvm -nographic -netdev tap,id=t0,ifname=tap0,script=no,downscript=no,vhost=on -device virtio-net-device,netdev=t0,id=nic0 \
>>> -kernel /extra/rootfs/boot/Image -drive file=/extra/rootfs.img,id=fs -device virtio-blk-device,drive=fs -m 512 -M virt -cpu host -append "console=ttyAMA0 console=ttyS0 root=/dev/vda"
>>>
>>>
>>> On my system -smp 2 or higher hangs in the guest kernel.
>> The -smp 2 hang issue is probably due to a missing PSCI v0.2 follow-up
>> patch to QEMU, you can try:
>> https://git.linaro.org/people/christoffer.dall/qemu-arm.git/shortlog/refs/heads/psci2-smp-fix
>>
>> [disclaimer: there may be a better fix somewhere on the qemu list, I
>> haven't kept track the last couple of days]
>>
>> -Christoffer
>

Re: KVM on ARM64

[Joel Schopp]

virtio will get you the best performance so why would you want to use 
something slower?

-Joel

On 08/07/2014 11:51 AM, Mathew Li wrote:
> Great. VirtIO works for me. Thanks for your help folks!
>
> Is there is any other way to add virtual disk, more like a traditional
> disk to qemu-system-aarch64? For example IDE disk or SATA disk or
> maybe as a SCSI disk?
>
> On Wed, Aug 6, 2014 at 9:48 AM, Joel Schopp <joel.schopp@amd.com> wrote:
>> It turns out that after a recent rebase of my kernel and qemu to the
>> latest the problem is fixed.  Rather than hunt down what fixed it I'm
>> just accepting the win and moving on. -smp 4 now works.
>>
>> -Joel
>>
>> On 08/06/2014 11:15 AM, Christoffer Dall wrote:
>>> On Tue, Aug 5, 2014 at 4:18 PM, Joel Schopp <joel.schopp@amd.com> wrote:
>>>> On 08/04/2014 07:35 PM, Mathew Li wrote:
>>>>> Hi,
>>>>>
>>>>> I have a quick question. How do we add a hard disk to the qemu ARM VM?
>>>>>
>>>>> I tried:
>>>>>
>>>>> qemu-system-aarch64 -machine virt -hda disk.img -kernel image -initrd initrd.img
>>>>>
>>>>> qemu-system-aarch64 -machine virt -sd disk.img -kernel image -initrd initrd.img
>>>>>
>>>>> qemu-system-aarch64 -machine virt -mtdblock disk.img -kernel image
>>>>> -initrd initrd.img
>>>>>
>>>>> Nothing seems to work. I am not able to see any disk (i.e. dev/sdX)
>>>>> inside guest OS.
>>>> I've been running something like this:
>>>>
>>>> qemu-system-aarch64 -smp 1 --enable-kvm -nographic -netdev tap,id=t0,ifname=tap0,script=no,downscript=no,vhost=on -device virtio-net-device,netdev=t0,id=nic0 \
>>>> -kernel /extra/rootfs/boot/Image -drive file=/extra/rootfs.img,id=fs -device virtio-blk-device,drive=fs -m 512 -M virt -cpu host -append "console=ttyAMA0 console=ttyS0 root=/dev/vda"
>>>>
>>>>
>>>> On my system -smp 2 or higher hangs in the guest kernel.
>>> The -smp 2 hang issue is probably due to a missing PSCI v0.2 follow-up
>>> patch to QEMU, you can try:
>>> https://git.linaro.org/people/christoffer.dall/qemu-arm.git/shortlog/refs/heads/psci2-smp-fix
>>>
>>> [disclaimer: there may be a better fix somewhere on the qemu list, I
>>> haven't kept track the last couple of days]
>>>
>>> -Christoffer

Re: KVM on ARM64

[Christoffer Dall]

That's probably because you updated your guest kernel to one that
supports PSCI v0.2 and therefore ignores the incorrect function IDs in
the DT (as it should).

I've sent a fix to qemu-devel@ today:
http://lists.gnu.org/archive/html/qemu-devel/2014-08/msg01179.html

On Wed, Aug 6, 2014 at 6:48 PM, Joel Schopp <joel.schopp@amd.com> wrote:
> It turns out that after a recent rebase of my kernel and qemu to the
> latest the problem is fixed.  Rather than hunt down what fixed it I'm
> just accepting the win and moving on. -smp 4 now works.
>
> -Joel
>
> On 08/06/2014 11:15 AM, Christoffer Dall wrote:
>> On Tue, Aug 5, 2014 at 4:18 PM, Joel Schopp <joel.schopp@amd.com> wrote:
>>> On 08/04/2014 07:35 PM, Mathew Li wrote:
>>>> Hi,
>>>>
>>>> I have a quick question. How do we add a hard disk to the qemu ARM VM?
>>>>
>>>> I tried:
>>>>
>>>> qemu-system-aarch64 -machine virt -hda disk.img -kernel image -initrd initrd.img
>>>>
>>>> qemu-system-aarch64 -machine virt -sd disk.img -kernel image -initrd initrd.img
>>>>
>>>> qemu-system-aarch64 -machine virt -mtdblock disk.img -kernel image
>>>> -initrd initrd.img
>>>>
>>>> Nothing seems to work. I am not able to see any disk (i.e. dev/sdX)
>>>> inside guest OS.
>>> I've been running something like this:
>>>
>>> qemu-system-aarch64 -smp 1 --enable-kvm -nographic -netdev tap,id=t0,ifname=tap0,script=no,downscript=no,vhost=on -device virtio-net-device,netdev=t0,id=nic0 \
>>> -kernel /extra/rootfs/boot/Image -drive file=/extra/rootfs.img,id=fs -device virtio-blk-device,drive=fs -m 512 -M virt -cpu host -append "console=ttyAMA0 console=ttyS0 root=/dev/vda"
>>>
>>>
>>> On my system -smp 2 or higher hangs in the guest kernel.
>> The -smp 2 hang issue is probably due to a missing PSCI v0.2 follow-up
>> patch to QEMU, you can try:
>> https://git.linaro.org/people/christoffer.dall/qemu-arm.git/shortlog/refs/heads/psci2-smp-fix
>>
>> [disclaimer: there may be a better fix somewhere on the qemu list, I
>> haven't kept track the last couple of days]
>>
>> -Christoffer
>

Re: KVM on ARM64

[Christoffer Dall]

Currently we only model a virtual machine board (the -machine
type=virt parameter) which has a UART, a flash, an RTC, and a bunch of
virtio-mmio channelse.

Once we either emulate a real aarch64 board (with whatever peripherals
it may have) or add a PCI controller to the virt board, then you can
choose whatever storage the real board has or start doing interesting
things like plugging in a scsi controller to your PCI controller on
the virt board or whatever else you desire.

But as Joel points out, VirtIO is likely to get you the best
performance and is the most convenient method.

-Christoffer

On Thu, Aug 7, 2014 at 6:51 PM, Mathew Li <mathew.li100@gmail.com> wrote:
> Great. VirtIO works for me. Thanks for your help folks!
>
> Is there is any other way to add virtual disk, more like a traditional
> disk to qemu-system-aarch64? For example IDE disk or SATA disk or
> maybe as a SCSI disk?
>

Re: KVM on ARM64

[Wei Huang]

On 08/07/2014 12:53 PM, Christoffer Dall wrote:
> Currently we only model a virtual machine board (the -machine
> type=virt parameter) which has a UART, a flash, an RTC, and a bunch of
> virtio-mmio channelse.
>
> Once we either emulate a real aarch64 board (with whatever peripherals
> it may have) or add a PCI controller to the virt board, then you can
Out of curiosity, any particular of PCI controller in your mind? I have 
seen people using Synopsys IP in real SOC; but wonder if anything else?

-Wei

> choose whatever storage the real board has or start doing interesting
> things like plugging in a scsi controller to your PCI controller on
> the virt board or whatever else you desire.
>
> But as Joel points out, VirtIO is likely to get you the best
> performance and is the most convenient method.
>
> -Christoffer
>
> On Thu, Aug 7, 2014 at 6:51 PM, Mathew Li <mathew.li100@gmail.com> wrote:
>> Great. VirtIO works for me. Thanks for your help folks!
>>
>> Is there is any other way to add virtual disk, more like a traditional
>> disk to qemu-system-aarch64? For example IDE disk or SATA disk or
>> maybe as a SCSI disk?
>>
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Re: KVM on ARM64

[Joel Schopp]

On 08/07/2014 12:53 PM, Christoffer Dall wrote:
> Currently we only model a virtual machine board (the -machine
> type=virt parameter) which has a UART, a flash, an RTC, and a bunch of
> virtio-mmio channelse.
>
> Once we either emulate a real aarch64 board (with whatever peripherals
> it may have) or add a PCI controller to the virt board, then you can
> choose whatever storage the real board has or start doing interesting
> things like plugging in a scsi controller to your PCI controller on
> the virt board or whatever else you desire.
I am very interested in having a PCI controller on the virt board to be 
able to do some testing of "-device pci-assign" and "-device vfio-pci".  
I noticed that Alvise Rigo (ccd) had sent some patches out to the 
qemu-devel list July 11th that seem to add a generic pci controller.

>
> But as Joel points out, VirtIO is likely to get you the best
> performance and is the most convenient method.
>
> -Christoffer
>
> On Thu, Aug 7, 2014 at 6:51 PM, Mathew Li <mathew.li100@gmail.com> wrote:
>> Great. VirtIO works for me. Thanks for your help folks!
>>
>> Is there is any other way to add virtual disk, more like a traditional
>> disk to qemu-system-aarch64? For example IDE disk or SATA disk or
>> maybe as a SCSI disk?
>>

Re: KVM on ARM64

[Christoffer Dall]

On Thu, Aug 07, 2014 at 01:06:09PM -0500, Wei Huang wrote:
> 
> 
> On 08/07/2014 12:53 PM, Christoffer Dall wrote:
> >Currently we only model a virtual machine board (the -machine
> >type=virt parameter) which has a UART, a flash, an RTC, and a bunch of
> >virtio-mmio channelse.
> >
> >Once we either emulate a real aarch64 board (with whatever peripherals
> >it may have) or add a PCI controller to the virt board, then you can
> Out of curiosity, any particular of PCI controller in your mind? I
> have seen people using Synopsys IP in real SOC; but wonder if
> anything else?
> 

We are looking at adding a generic PCI controller (which is what Alvise
is also working on).  Will Deacon did the original work for ARM and
Liviu Dudau is doing the ARM64 version (look in lakml).

-Christoffer