From: John Fastabend <john.fastabend@gmail.com>
To: "Michael S. Tsirkin" <mst@redhat.com>, Jason Wang <jasowang@redhat.com>
Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: Re: [PATCH 1/3] tuntap: rx batching
Date: Thu, 10 Nov 2016 20:17:24 -0800 [thread overview]
Message-ID: <58254654.4000501@gmail.com> (raw)
In-Reply-To: <20161111053048-mutt-send-email-mst@kernel.org>
On 16-11-10 07:31 PM, Michael S. Tsirkin wrote:
> On Fri, Nov 11, 2016 at 10:07:44AM +0800, Jason Wang wrote:
>>
>>
>> On 2016年11月10日 00:38, Michael S. Tsirkin wrote:
>>> On Wed, Nov 09, 2016 at 03:38:31PM +0800, Jason Wang wrote:
>>>> Backlog were used for tuntap rx, but it can only process 1 packet at
>>>> one time since it was scheduled during sendmsg() synchronously in
>>>> process context. This lead bad cache utilization so this patch tries
>>>> to do some batching before call rx NAPI. This is done through:
>>>>
>>>> - accept MSG_MORE as a hint from sendmsg() caller, if it was set,
>>>> batch the packet temporarily in a linked list and submit them all
>>>> once MSG_MORE were cleared.
>>>> - implement a tuntap specific NAPI handler for processing this kind of
>>>> possible batching. (This could be done by extending backlog to
>>>> support skb like, but using a tun specific one looks cleaner and
>>>> easier for future extension).
>>>>
>>>> Signed-off-by: Jason Wang <jasowang@redhat.com>
>>> So why do we need an extra queue?
>>
>> The idea was borrowed from backlog to allow some kind of bulking and avoid
>> spinlock on each dequeuing.
>>
>>> This is not what hardware devices do.
>>> How about adding the packet to queue unconditionally, deferring
>>> signalling until we get sendmsg without MSG_MORE?
>>
>> Then you need touch spinlock when dequeuing each packet.
>
Random thought, I have a cmpxchg ring I am using for the qdisc work that
could possibly replace the spinlock implementation. I haven't figured
out the resizing API yet because I did not need it but I assume it could
help here and let you dequeue multiple skbs in one operation.
I can post the latest version if useful or an older version is
somewhere on patchworks as well.
.John
> It runs on the same CPU, right? Otherwise we should use skb_array...
>
>>>
>>>
>>>> ---
>>>> drivers/net/tun.c | 71 ++++++++++++++++++++++++++++++++++++++++++++++++++-----
>>>> 1 file changed, 65 insertions(+), 6 deletions(-)
>>>>
>>
>> [...]
>>
>>>> rxhash = skb_get_hash(skb);
>>>> - netif_rx_ni(skb);
>>>> + skb_queue_tail(&tfile->socket.sk->sk_write_queue, skb);
>>>> +
>>>> + if (!more) {
>>>> + local_bh_disable();
>>>> + napi_schedule(&tfile->napi);
>>>> + local_bh_enable();
>>> Why do we need to disable bh here? I thought napi_schedule can
>>> be called from any context.
>>
>> Yes, it's unnecessary. Will remove.
>>
>> Thanks
next prev parent reply other threads:[~2016-11-11 4:17 UTC|newest]
Thread overview: 25+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-11-09 7:38 [PATCH 1/3] tuntap: rx batching Jason Wang
2016-11-09 7:38 ` [PATCH 2/3] vhost: better detection of available buffers Jason Wang
2016-11-09 19:57 ` Michael S. Tsirkin
2016-11-11 2:18 ` Jason Wang
2016-11-11 3:41 ` Michael S. Tsirkin
2016-11-11 4:18 ` Jason Wang
2016-11-11 16:20 ` Michael S. Tsirkin
2016-11-15 3:16 ` Jason Wang
2016-11-15 3:28 ` Michael S. Tsirkin
2016-11-15 8:00 ` Jason Wang
2016-11-15 14:46 ` Michael S. Tsirkin
2016-11-09 7:38 ` [PATCH 3/3] vhost_net: tx support batching Jason Wang
2016-11-09 20:05 ` Michael S. Tsirkin
2016-11-11 2:27 ` Jason Wang
2016-11-09 16:38 ` [PATCH 1/3] tuntap: rx batching Michael S. Tsirkin
2016-11-11 2:07 ` Jason Wang
2016-11-11 3:31 ` Michael S. Tsirkin
2016-11-11 4:10 ` Jason Wang
2016-11-11 4:17 ` John Fastabend [this message]
2016-11-11 4:28 ` Jason Wang
2016-11-11 4:45 ` John Fastabend
2016-11-11 16:20 ` Michael S. Tsirkin
2016-11-15 3:14 ` Jason Wang
2016-11-15 3:41 ` Michael S. Tsirkin
2016-11-15 8:08 ` Jason Wang
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=58254654.4000501@gmail.com \
--to=john.fastabend@gmail.com \
--cc=jasowang@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mst@redhat.com \
--cc=netdev@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).