From: Pankaj Gupta <pagupta@redhat.com>
To: linux-kernel@vger.kernel.org, netdev@vger.kernel.org
Cc: davem@davemloft.net, jasowang@redhat.com, mst@redhat.com,
dgibson@redhat.com, vfalico@gmail.com, edumazet@google.com,
vyasevic@redhat.com, hkchu@google.com, wuzhy@linux.vnet.ibm.com,
xemul@parallels.com, therbert@google.com,
bhutchings@solarflare.com, xii@google.com,
stephen@networkplumber.org, Pankaj Gupta <pagupta@redhat.com>
Subject: [RFC 0/4] Increase the limit of tuntap queues
Date: Mon, 18 Aug 2014 19:07:16 +0530 [thread overview]
Message-ID: <1408369040-1216-1-git-send-email-pagupta@redhat.com> (raw)
Networking under KVM works best if we allocate a per-vCPU rx and tx
queue in a virtual NIC. This requires a per-vCPU queue on the host side.
Modern physical NICs have multiqueue support for large number of queues.
To scale vNIC to run multiple queues parallel to maximum number of vCPU's
we need to increase number of queues support in tuntap.
This series is to increase the limit of tuntap queues. Original work is being
done by 'jasowang@redhat.com'. I am taking this 'https://lkml.org/lkml/2013/6/19/29'
patch series as a reference. As per discussion in the patch series:
There were two reasons which prevented us from increasing number of tun queues:
- The netdev_queue array in netdevice were allocated through kmalloc, which may
cause a high order memory allocation too when we have several queues.
E.g. sizeof(netdev_queue) is 320, which means a high order allocation would
happens when the device has more than 16 queues.
- We store the hash buckets in tun_struct which results a very large size of
tun_struct, this high order memory allocation fail easily when the memory is
fragmented.
The patch 60877a32bce00041528576e6b8df5abe9251fa73 increases the number of tx
queues. Memory allocation fallback to vzalloc() when kmalloc() fails.
This series tries to address following issues:
- Increase the number of netdev_queue queues for rx similarly its done for tx
queues by falling back to vzalloc() when memory allocation with kmalloc() fails.
- Switches to use flex array to implement the flow caches to avoid higher order
allocations.
- Publish maximum number of queues as read only module_param so that user space
application like libvirt can use this value to limit number of queues. Also
Administrators can specify number of queues at module load time.
- Increase number of queues to 256, maximum number is equal to maximum number
of vCPUS allowed in a guest.
I have done some testing to find out any regression and with sample program
which creates tun/tap for single queue / multiqueue device and it seems to be
working fine.
tuntap: Increase the number of queues in tun
tuntap: Reduce the size of tun_struct by using flex array
tuntap: Publish tuntap max queue length as module_param
net: allow large number of rx queues
drivers/net/tun.c | 71 ++++++++++++++++++++++++++++++++++++++++--------------
net/core/dev.c | 20 +++++++++------
2 files changed, 66 insertions(+), 25 deletions(-)
next reply other threads:[~2014-08-18 13:37 UTC|newest]
Thread overview: 19+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-08-18 13:37 Pankaj Gupta [this message]
2014-08-18 13:37 ` [RFC 1/4] net: allow large number of rx queues Pankaj Gupta
2014-08-18 17:43 ` Sergei Shtylyov
2014-08-19 5:15 ` Pankaj Gupta
2014-08-18 13:37 ` [RFC 2/4] tuntap: Publish tuntap maximum number of queues as module_param Pankaj Gupta
2014-08-20 10:58 ` Jiri Pirko
2014-08-20 11:17 ` Michael S. Tsirkin
2014-08-20 11:46 ` Jiri Pirko
2014-08-20 11:49 ` Michael S. Tsirkin
2014-08-20 11:53 ` Jiri Pirko
2014-08-21 4:30 ` Jason Wang
2014-08-22 11:52 ` Pankaj Gupta
2014-08-24 11:14 ` Michael S. Tsirkin
2014-08-25 2:57 ` Jason Wang
2014-08-26 15:30 ` Pankaj Gupta
2014-08-18 13:37 ` [RFC 3/4] tuntap: reduce the size of tun_struct by using flex array Pankaj Gupta
2014-08-25 1:35 ` David Gibson
2014-08-18 13:37 ` [RFC 4/4] tuntap: Increase the number of queues in tun Pankaj Gupta
2014-08-25 1:37 ` David Gibson
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1408369040-1216-1-git-send-email-pagupta@redhat.com \
--to=pagupta@redhat.com \
--cc=bhutchings@solarflare.com \
--cc=davem@davemloft.net \
--cc=dgibson@redhat.com \
--cc=edumazet@google.com \
--cc=hkchu@google.com \
--cc=jasowang@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mst@redhat.com \
--cc=netdev@vger.kernel.org \
--cc=stephen@networkplumber.org \
--cc=therbert@google.com \
--cc=vfalico@gmail.com \
--cc=vyasevic@redhat.com \
--cc=wuzhy@linux.vnet.ibm.com \
--cc=xemul@parallels.com \
--cc=xii@google.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.