Re: [Qemu-devel] [PATCH 10/12] virtio-net: multiqueue support

From: Jason Wang <jasowang@redhat.com>
To: gaowanlong@cn.fujitsu.com
Cc: krkumar2@in.ibm.com, aliguori@us.ibm.com, kvm@vger.kernel.org,
	mst@redhat.com, mprivozn@redhat.com, rusty@rustcorp.com.au,
	qemu-devel@nongnu.org, stefanha@redhat.com,
	jwhan@filewood.snu.ac.kr, shiyer@redhat.com
Subject: Re: [Qemu-devel] [PATCH 10/12] virtio-net: multiqueue support
Date: Wed, 09 Jan 2013 11:11:17 +0800	[thread overview]
Message-ID: <50ECDFD5.6010109@redhat.com> (raw)
In-Reply-To: <50EC01E0.8010102@cn.fujitsu.com>

On 01/08/2013 07:24 PM, Wanlong Gao wrote:
> On 01/08/2013 06:14 PM, Jason Wang wrote:
>> On 01/08/2013 06:00 PM, Wanlong Gao wrote:
>>> On 01/08/2013 05:51 PM, Jason Wang wrote:
>>>> On 01/08/2013 05:49 PM, Wanlong Gao wrote:
>>>>> On 01/08/2013 05:29 PM, Jason Wang wrote:
>>>>>> On 01/08/2013 05:07 PM, Wanlong Gao wrote:
>>>>>>> On 12/28/2012 06:32 PM, Jason Wang wrote:
>>>>>>>> +    } else if (nc->peer->info->type !=  NET_CLIENT_OPTIONS_KIND_TAP) {
>>>>>>>> +        ret = -1;
>>>>>>>> +    } else {
>>>>>>>> +        ret = tap_detach(nc->peer);
>>>>>>>> +    }
>>>>>>>> +
>>>>>>>> +    return ret;
>>>>>>>> +}
>>>>>>>> +
>>>>>>>> +static void virtio_net_set_queues(VirtIONet *n)
>>>>>>>> +{
>>>>>>>> +    int i;
>>>>>>>> +
>>>>>>>> +    for (i = 0; i < n->max_queues; i++) {
>>>>>>>> +        if (i < n->curr_queues) {
>>>>>>>> +            assert(!peer_attach(n, i));
>>>>>>>> +        } else {
>>>>>>>> +            assert(!peer_detach(n, i));
>>>>>>> I got a assert here,
>>>>>>> qemu-system-x86_64: /work/git/qemu/hw/virtio-net.c:330: virtio_net_set_queues: Assertion `!peer_detach(n, i)' failed.
>>>>>>>
>>>>>>> Any thoughts?
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Wanlong Gao
>>>>>> Thanks for the testing, which steps or cases did you met this assertion,
>>>>>> migration, reboot or just changing the number of virtqueues?
>>>>> I use the 3.8-rc2 to test it again, I saw this tag has the multi-tap support.
>>>>>
>>>>> I just can't start the QEMU use  -netdev tap,id=hostnet0,queues=2,fd=%d,fd=%d -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:ce:7b:29,bus=pci.0,addr=0x3
>>>>>
>>>>> I pre-opened two tap fds, did I missing something?
>>>> Nothing missed :) It should work.
>>>>
>>>> Could you please try not use fd=X and let qemu to create the file
>>>> descriptors by itself? Btw, how did you create the two tap fds?
>>> Can it create descriptors itself? I get 
>>> qemu-system-x86_64: -netdev tap,id=hostnet0,queues=2: Device 'tap' could not be initialized
>> You need prepare an ifup script which default at /etc/qemu-ifup (like
>> following). Or you may try to add a script=no after:
>>
>> #!/bin/sh
>>
>> switch=kvmbr0
>>
>> /sbin/ifconfig $1 0.0.0.0 up
>> /usr/sbin/brctl addif $switch $1
>> /usr/sbin/brctl stp $switch off
>>
>> This will let qemu create a tap fd itself and make it to be connected to
>> a port of the bridge caled kvmbr0.
>>> I create the tap fd like this, and dup create the second fd, third fd, right?
>> The second and third fd should be created with TUNSETIFF with the same
>> tap_name also. Btw, you need to specify a IFF_MULTI_QUEUE flag to tell
>> the kernel you want to create a multiqueue tap device, otherwise the
>> second and third calling of TUNSETIFF will fail.
> Thank you for teaching me, I'll try it tomorrow.
>
> Regards,
> Wanlong Gao

Thanks, the API of multiqueue should be documented in
Documentation/networking/tuntap.txt. It's in my TODO list.
>
>> Thanks
>>> 	int tap_fd = open("/dev/net/tun", O_RDWR);
>>> 	int vhost_fd = open("/dev/vhost-net", O_RDWR);
>>> 	char *tap_name = "tap";
>>> 	char cmd[2048];
>>> 	char brctl[256];
>>> 	char netup[256];
>>> 	struct ifreq ifr;
>>> 	if (tap_fd < 0) {
>>> 		printf("open tun device failed\n");
>>> 		return -1;
>>> 	}
>>> 	if (vhost_fd < 0) {
>>> 		printf("open vhost-net device failed\n");
>>> 		return -1;
>>> 	}
>>> 	memset(&ifr, 0, sizeof(ifr));
>>> 	memcpy(ifr.ifr_name, tap_name, sizeof(tap_name));
>>> 	ifr.ifr_flags = IFF_TAP | IFF_NO_PI;
>>>
>>> 	/*
>>> 	 * setup tap net device
>>> 	 */
>>> 	if (ioctl(tap_fd, TUNSETIFF, &ifr) < 0) {
>>> 		printf("setup tap net device failed\n");
>>> 		return -1;
>>> 	}
>>>
>>> 	sprintf(brctl, "brctl addif virbr0 %s", tap_name);
>>> 	sprintf(netup, "ifconfig %s up", tap_name);
>>> 	system(brctl);
>>> 	system(netup);
>>>
>>> Thanks,
>>> Wanlong Gao
>>>
>>>
>>>> Thanks
>>>>> Thanks,
>>>>> Wanlong Gao
>>>>>
>>>>>>>> +        }
>>>>>>>> +    }
>>>>>>>> +}
>>>>>>>> +
>>>>>>>> +static void virtio_net_set_multiqueue(VirtIONet *n, int multiqueue, int ctrl);
>>>>>>>> +
>>>>>>> --
>>>>>>> To unsubscribe from this list: send the line "unsubscribe kvm" in
>>>>>>> the body of a message to majordomo@vger.kernel.org
>>>>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html