From: Jason Wang <jasowang@redhat.com>
To: "Michael S. Tsirkin" <mst@redhat.com>
Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org,
virtualization@lists.linux-foundation.org, pagupta@redhat.com
Subject: Re: [PATCH RFC v5 net-next 2/6] virtio_ring: try to disable event index callbacks in virtqueue_disable_cb()
Date: Wed, 11 Feb 2015 06:03:36 +0008 [thread overview]
Message-ID: <1423634136.4369.1@smtp.corp.redhat.com> (raw)
In-Reply-To: <20150210102417.GB9505@redhat.com>
On Tue, Feb 10, 2015 at 6:24 PM, Michael S. Tsirkin <mst@redhat.com>
wrote:
> On Mon, Feb 09, 2015 at 03:39:21AM -0500, Jason Wang wrote:
>> Currently, we do nothing to prevent the callbacks in
>> virtqueue_disable_cb() when event index is used. This may cause
>> spurious interrupts which may damage the performance. This patch
>> tries
>> to publish avail event as the used even to prevent the callbacks.
>>
>> Signed-off-by: Jason Wang <jasowang@redhat.com>
>
> I'm surprised that this ever happens though.
> Normally we call this after getting an interrupt, and
> interrupts won't trigger again until the rings wraps around.
It was used to disable tx interrupt during start_xmit().
>
> When I tested this, touching an extra cache line was more
> expensive.
>
> Does this really reduce number of interrupts?
> Could you pls share some numbers with and without this patch?
Yes. It does reduce, see the following test results on multiple
sessions of TCP_RR
Datacopy:
/size/sessions/+-throughput%/+-cpu%/+-per_cpu_throughput%/+-tx_interrupts%/
/1/50/+0.1/+0.7/-0.6/-72.3/
/64/50/+1.7/+1.0/+0.7/-72.2/
/256/50/+0.0/-0.8/+0.9/-72.4/
Zerocopy:
/size/sessions/+-throughput%/+-cpu%/+-per_cpu_throughput%/+-tx_interrupts%/
/1/50/+3.2/-1.3/+4.6/-71.7/
/64/50/+1.9/-0.2/+2.1/-70.8/
/256/50/+4.4/+0.6/+3.9/-84.2/
No obvious changes for stream rx and tx.
>
>
>
>> ---
>> drivers/virtio/virtio_ring.c | 2 ++
>> 1 file changed, 2 insertions(+)
>>
>> diff --git a/drivers/virtio/virtio_ring.c
>> b/drivers/virtio/virtio_ring.c
>> index 545fed5..e9ffbfb 100644
>> --- a/drivers/virtio/virtio_ring.c
>> +++ b/drivers/virtio/virtio_ring.c
>> @@ -539,6 +539,8 @@ void virtqueue_disable_cb(struct virtqueue *_vq)
>> struct vring_virtqueue *vq = to_vvq(_vq);
>>
>> vq->vring.avail->flags |= cpu_to_virtio16(_vq->vdev,
>> VRING_AVAIL_F_NO_INTERRUPT);
>> + vring_used_event(&vq->vring) = cpu_to_virtio16(_vq->vdev,
>> + vq->vring.avail->idx);
>
> Hmm in fact, can't this actually cause an extra interrupt
> when avail->idx is completed?
We need to try best to avoid any interrupt after this and
virtqueue_disable_cb() were always need virtqueue_enable_cb() or
virtqueue_enable_cb_delayed() afterwards. Those two function will check
the pending used buffers and set a proper used event.
>
>
> I think that if you really can show disabling interrupts like this
> helps, you should
> set some invalid value like 0xfffff, or move it back to
> vq->vring.avail->idx - 1.
> No?
I tested avail->idx + vq.num but not obvious changes in the performance
compared to just use avail->idx here. So 0xffff probably won't help
much. Since we call virtqueue_enable_cb() or
virtqueue_disable_cb_delayed() afterwards, it doesn't matter that
whether avail->idx or avail->idx - 1 is used.
>
>
>
>
>> }
>> EXPORT_SYMBOL_GPL(virtqueue_disable_cb);
>>
>> --
>> 1.8.3.1
> --
> To unsubscribe from this list: send the line "unsubscribe netdev" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
next prev parent reply other threads:[~2015-02-11 5:55 UTC|newest]
Thread overview: 24+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-02-09 8:39 [PATCH RFC v5 net-next 0/6] enable tx interrupts for virtio-net Jason Wang
2015-02-09 8:39 ` [PATCH RFC v5 net-next 1/6] virtio_ring: fix virtqueue_enable_cb() when only 1 buffers were pending Jason Wang
2015-02-10 1:03 ` Rusty Russell
2015-02-10 6:26 ` Jason Wang
2015-02-10 10:18 ` Michael S. Tsirkin
2015-02-10 23:58 ` Rusty Russell
2015-02-11 5:41 ` Jason Wang
2015-02-09 8:39 ` [PATCH RFC v5 net-next 2/6] virtio_ring: try to disable event index callbacks in virtqueue_disable_cb() Jason Wang
2015-02-10 1:07 ` Rusty Russell
2015-02-10 10:24 ` Michael S. Tsirkin
2015-02-11 5:55 ` Jason Wang [this message]
2015-02-09 8:39 ` [PATCH RFC v5 net-next 3/6] virtio_net: enable tx interrupt Jason Wang
2015-02-09 8:39 ` [PATCH RFC v5 net-next 4/6] virtio-net: add basic interrupt coalescing support Jason Wang
2015-02-10 1:32 ` Rusty Russell
2015-02-10 6:51 ` Jason Wang
2015-02-10 10:25 ` Michael S. Tsirkin
2015-02-10 10:40 ` Michael S. Tsirkin
2015-02-13 2:52 ` Rusty Russell
2015-02-13 12:41 ` Pawel Moll
2015-02-16 3:07 ` Rusty Russell
2015-02-13 18:19 ` Cornelia Huck
2015-02-16 3:19 ` Rusty Russell
2015-02-09 8:39 ` [PATCH RFC v5 net-next 5/6] vhost: let vhost_signal() returns whether signalled Jason Wang
2015-02-09 8:39 ` [PATCH RFC v5 net-next 6/6] vhost_net: interrupt coalescing support Jason Wang
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1423634136.4369.1@smtp.corp.redhat.com \
--to=jasowang@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mst@redhat.com \
--cc=netdev@vger.kernel.org \
--cc=pagupta@redhat.com \
--cc=virtualization@lists.linux-foundation.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).