From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Daniel P. Berrange" Subject: Re: [Qemu-devel] Hitting 29 NIC limit Date: Thu, 14 Oct 2010 13:10:47 +0100 Message-ID: <20101014121047.GN11403@redhat.com> References: <4CB6388A.30006@codemonkey.ws> <4CB6F275.2060204@redhat.com> Reply-To: "Daniel P. Berrange" Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: Anthony Liguori , Anjali Kulkarni , "kvm@vger.kernel.org" , "qemu-devel@nongnu.org" To: Avi Kivity Return-path: Received: from mx1.redhat.com ([209.132.183.28]:30434 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752342Ab0JNMLP (ORCPT ); Thu, 14 Oct 2010 08:11:15 -0400 Content-Disposition: inline In-Reply-To: <4CB6F275.2060204@redhat.com> Sender: kvm-owner@vger.kernel.org List-ID: On Thu, Oct 14, 2010 at 02:07:17PM +0200, Avi Kivity wrote: > On 10/14/2010 12:54 AM, Anthony Liguori wrote: > >On 10/13/2010 05:32 PM, Anjali Kulkarni wrote: > >> > >>Hi, > >> > >>Using the legacy way of starting up NICs, I am hitting a limitation > >>after 29 > >>NICs ie no more than 29 are detected (that's because of the 32 PCI slot > >>limit on a single bus- 3 are already taken up) > >>I had initially increased the MAX_NICS to 48, just on my tree, to get to > >>more, but ofcource that wont work. > >>Is there any way to go beyond 29 NICs the legacy way? What is the > >>maximum > >>that can be supported by the qdev mothod? > > > >I got up to 104 without trying very hard using the following script: > > > >args="" > >for slot in 5 6 7 8 9 10 11 12 13 14 15 16 17; do > >for fn in 0 1 2 3 4 5 6 7; do > > args="$args -netdev user,id=eth${slot}_${fn}" > > args="$args -device > >virtio-net-pci,addr=${slot}.${fn},netdev=eth${slot}_${fn},multifunction=on,romfile=" > >done > >done > > > >x86_64-softmmu/qemu-system-x86_64 -hda ~/images/linux.img ${args} > >-enable-kvm > > > >The key is to make the virtio-net devices multifunction and to fill > >out all 8 functions for each slot. > > This is unlikely to work right wrt pci hotplug. If we want to support a > large number of interfaces, we need true multiport cards. Or a PCI bridge to wire up more PCI buses, so we raise the max limit for any type of device we emulate. Daniel -- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://deltacloud.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :| From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from [140.186.70.92] (port=58815 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1P6MeT-0000XI-1e for qemu-devel@nongnu.org; Thu, 14 Oct 2010 08:11:18 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1P6MeR-00019z-Sc for qemu-devel@nongnu.org; Thu, 14 Oct 2010 08:11:16 -0400 Received: from mx1.redhat.com ([209.132.183.28]:26825) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1P6MeR-00019R-MC for qemu-devel@nongnu.org; Thu, 14 Oct 2010 08:11:15 -0400 Date: Thu, 14 Oct 2010 13:10:47 +0100 From: "Daniel P. Berrange" Subject: Re: [Qemu-devel] Hitting 29 NIC limit Message-ID: <20101014121047.GN11403@redhat.com> References: <4CB6388A.30006@codemonkey.ws> <4CB6F275.2060204@redhat.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4CB6F275.2060204@redhat.com> Reply-To: "Daniel P. Berrange" List-Id: qemu-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Avi Kivity Cc: Anjali Kulkarni , "kvm@vger.kernel.org" , "qemu-devel@nongnu.org" On Thu, Oct 14, 2010 at 02:07:17PM +0200, Avi Kivity wrote: > On 10/14/2010 12:54 AM, Anthony Liguori wrote: > >On 10/13/2010 05:32 PM, Anjali Kulkarni wrote: > >> > >>Hi, > >> > >>Using the legacy way of starting up NICs, I am hitting a limitation > >>after 29 > >>NICs ie no more than 29 are detected (that's because of the 32 PCI slot > >>limit on a single bus- 3 are already taken up) > >>I had initially increased the MAX_NICS to 48, just on my tree, to get to > >>more, but ofcource that wont work. > >>Is there any way to go beyond 29 NICs the legacy way? What is the > >>maximum > >>that can be supported by the qdev mothod? > > > >I got up to 104 without trying very hard using the following script: > > > >args="" > >for slot in 5 6 7 8 9 10 11 12 13 14 15 16 17; do > >for fn in 0 1 2 3 4 5 6 7; do > > args="$args -netdev user,id=eth${slot}_${fn}" > > args="$args -device > >virtio-net-pci,addr=${slot}.${fn},netdev=eth${slot}_${fn},multifunction=on,romfile=" > >done > >done > > > >x86_64-softmmu/qemu-system-x86_64 -hda ~/images/linux.img ${args} > >-enable-kvm > > > >The key is to make the virtio-net devices multifunction and to fill > >out all 8 functions for each slot. > > This is unlikely to work right wrt pci hotplug. If we want to support a > large number of interfaces, we need true multiport cards. Or a PCI bridge to wire up more PCI buses, so we raise the max limit for any type of device we emulate. Daniel -- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://deltacloud.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|