Issues with vhostfd when calling qemu-kvm directly

Issues with vhostfd when calling qemu-kvm directly

[Roberto Cardona]

I'm trying to start a virtual machine by calling qemu-kvm directly but
I am having issues with the vhost file descriptor (vhostfd).

According to qemu's source code you must pass the file descriptor of
the previously opened vhost_net device so, in order to do this, I am
using a python script with something like this: os.open(
"/dev/vhost-net", os.O_RDWR). Afterwards it passes this to qemu-kvm as
a subprocess.

I took this off the command that libvirt generates so I know this work
and all kernel modules are enabled.

Is there any workaround for this so that I can call qemu-kvm directly
without libvirt?

Error messages:
qemu-kvm: -netdev tap,id=hostnet0,vhost=on,fd=3: TUNGETIFF ioctl()
failed: Inappropriate ioctl for device
qemu-kvm: -netdev tap,id=hostnet1,vhost=on,fd=4: TUNGETIFF ioctl()
failed: Inappropriate ioctl for device

Network device arguments passed to qemu-kvm (fd 3 and 4 are the fds
that Python opened):
-netdev tap,id=hostnet0,vhost=on,fd=3 3<>/dev/tap5,vhostfd=3
-device virtio-net
pci,netdev=hostnet0,id=net0,mac=52:54:00:01:cb:a5,bus=pci.0,addr=0x3
-netdev tap,id=hostnet1,vhost=on,fd=4 4<>/dev/tap24,vhostfd=4
-device virtio-net
pci,netdev=hostnet1,id=net1,mac=52:50:00:1b:ce:6e,bus=pci.0,addr=0x4

Version:
QEMU emulator version 2.6.0 (qemu-kvm-ev-2.6.0-28.el7_3.3.1)

Thank you.

Re: Issues with vhostfd when calling qemu-kvm directly

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 2066 bytes --]

On Fri, Feb 24, 2017 at 08:37:51AM -0800, Roberto Cardona wrote:
> I'm trying to start a virtual machine by calling qemu-kvm directly but
> I am having issues with the vhost file descriptor (vhostfd).
> 
> According to qemu's source code you must pass the file descriptor of
> the previously opened vhost_net device so, in order to do this, I am
> using a python script with something like this: os.open(
> "/dev/vhost-net", os.O_RDWR). Afterwards it passes this to qemu-kvm as
> a subprocess.
> 
> I took this off the command that libvirt generates so I know this work
> and all kernel modules are enabled.
> 
> Is there any workaround for this so that I can call qemu-kvm directly
> without libvirt?
> 
> Error messages:
> qemu-kvm: -netdev tap,id=hostnet0,vhost=on,fd=3: TUNGETIFF ioctl()
> failed: Inappropriate ioctl for device
> qemu-kvm: -netdev tap,id=hostnet1,vhost=on,fd=4: TUNGETIFF ioctl()
> failed: Inappropriate ioctl for device
> 
> Network device arguments passed to qemu-kvm (fd 3 and 4 are the fds
> that Python opened):
> -netdev tap,id=hostnet0,vhost=on,fd=3 3<>/dev/tap5,vhostfd=3
> -device virtio-net
> pci,netdev=hostnet0,id=net0,mac=52:54:00:01:cb:a5,bus=pci.0,addr=0x3
> -netdev tap,id=hostnet1,vhost=on,fd=4 4<>/dev/tap24,vhostfd=4
> -device virtio-net
> pci,netdev=hostnet1,id=net1,mac=52:50:00:1b:ce:6e,bus=pci.0,addr=0x4
> 
> Version:
> QEMU emulator version 2.6.0 (qemu-kvm-ev-2.6.0-28.el7_3.3.1)

File descriptor passing is optional.  Libvirt uses it so that QEMU can
be launched without root privileges.  It's a good idea to follow this
practice for production environments or when dealing with untrusted
guests.  But remember, it's optional and you can skip fd passing if you
want.

fd=X and vhostfd=Y are supposed to be different device types:
 * fd=X is a tap-like device (either /dev/tapN or macvtap)
 * vhostfd=Y is /dev/vhost-net

Looks like your code is just opening /dev/tapN and using that fd for
both fd=X and vhostfd=Y.  This is incorrect and causes the ioctl error.

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 455 bytes --]

Re: Issues with vhostfd when calling qemu-kvm directly

[R]

Thank you Stefan! I figured it out and it now works.
I read on your blog that one can pass QEMU commands on libvirt
(http://blog.vmsplice.net/2011/04/how-to-pass-qemu-command-line-options.html).
Would it be possible to fully define a network interface through this
and there completely avoiding the libvirt commands for network
interfaces?
Thank you.


On Mon, Feb 27, 2017 at 2:04 AM, Stefan Hajnoczi <stefanha@gmail.com> wrote:
> On Fri, Feb 24, 2017 at 08:37:51AM -0800, Roberto Cardona wrote:
>> I'm trying to start a virtual machine by calling qemu-kvm directly but
>> I am having issues with the vhost file descriptor (vhostfd).
>>
>> According to qemu's source code you must pass the file descriptor of
>> the previously opened vhost_net device so, in order to do this, I am
>> using a python script with something like this: os.open(
>> "/dev/vhost-net", os.O_RDWR). Afterwards it passes this to qemu-kvm as
>> a subprocess.
>>
>> I took this off the command that libvirt generates so I know this work
>> and all kernel modules are enabled.
>>
>> Is there any workaround for this so that I can call qemu-kvm directly
>> without libvirt?
>>
>> Error messages:
>> qemu-kvm: -netdev tap,id=hostnet0,vhost=on,fd=3: TUNGETIFF ioctl()
>> failed: Inappropriate ioctl for device
>> qemu-kvm: -netdev tap,id=hostnet1,vhost=on,fd=4: TUNGETIFF ioctl()
>> failed: Inappropriate ioctl for device
>>
>> Network device arguments passed to qemu-kvm (fd 3 and 4 are the fds
>> that Python opened):
>> -netdev tap,id=hostnet0,vhost=on,fd=3 3<>/dev/tap5,vhostfd=3
>> -device virtio-net
>> pci,netdev=hostnet0,id=net0,mac=52:54:00:01:cb:a5,bus=pci.0,addr=0x3
>> -netdev tap,id=hostnet1,vhost=on,fd=4 4<>/dev/tap24,vhostfd=4
>> -device virtio-net
>> pci,netdev=hostnet1,id=net1,mac=52:50:00:1b:ce:6e,bus=pci.0,addr=0x4
>>
>> Version:
>> QEMU emulator version 2.6.0 (qemu-kvm-ev-2.6.0-28.el7_3.3.1)
>
> File descriptor passing is optional.  Libvirt uses it so that QEMU can
> be launched without root privileges.  It's a good idea to follow this
> practice for production environments or when dealing with untrusted
> guests.  But remember, it's optional and you can skip fd passing if you
> want.
>
> fd=X and vhostfd=Y are supposed to be different device types:
>  * fd=X is a tap-like device (either /dev/tapN or macvtap)
>  * vhostfd=Y is /dev/vhost-net
>
> Looks like your code is just opening /dev/tapN and using that fd for
> both fd=X and vhostfd=Y.  This is incorrect and causes the ioctl error.
>
> Stefan

Re: Issues with vhostfd when calling qemu-kvm directly

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 680 bytes --]

On Mon, Feb 27, 2017 at 06:47:04PM -0800, R wrote:
> Thank you Stefan! I figured it out and it now works.
> I read on your blog that one can pass QEMU commands on libvirt
> (http://blog.vmsplice.net/2011/04/how-to-pass-qemu-command-line-options.html).
> Would it be possible to fully define a network interface through this
> and there completely avoiding the libvirt commands for network
> interfaces?

Yes.  Network setup is one of the main conveniences that libvirt offers
over invoking QEMU manually.  I'd check again whether you can use
libvirt's network settings or maybe just invoke QEMU manually if you
have a specialized use-case where you'd be fighting libvirt.

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 455 bytes --]