From: Eric Dumazet <eric.dumazet@gmail.com> To: "Michael S. Tsirkin" <mst@redhat.com>, linux-kernel@vger.kernel.org Cc: Jakub Kicinski <kuba@kernel.org>, Wei Wang <weiwan@google.com>, David Miller <davem@davemloft.net>, netdev@vger.kernel.org, Willem de Bruijn <willemb@google.com>, virtualization@lists.linux-foundation.org, Jason Wang <jasowang@redhat.com> Subject: Re: [PATCH v3 4/4] virtio_net: disable cb aggressively Date: Wed, 26 May 2021 17:15:08 +0200 [thread overview] Message-ID: <18b47c22-8c8a-7699-ffaf-ccfdcbf39d37@gmail.com> (raw) In-Reply-To: <20210526082423.47837-5-mst@redhat.com> On 5/26/21 10:24 AM, Michael S. Tsirkin wrote: > There are currently two cases where we poll TX vq not in response to a > callback: start xmit and rx napi. We currently do this with callbacks > enabled which can cause extra interrupts from the card. Used not to be > a big issue as we run with interrupts disabled but that is no longer the > case, and in some cases the rate of spurious interrupts is so high > linux detects this and actually kills the interrupt. > > Fix up by disabling the callbacks before polling the tx vq. It is not clear why we want to poll TX completions from ndo_start_xmit() in napi mode ? This seems not needed, adding costs to sender thread, this might reduce the ability to use a different cpu for tx completions. Also this will likely conflict with BQL model if we want to use BQL at some point. > This probably needs a Fixes: tag > Signed-off-by: Michael S. Tsirkin <mst@redhat.com> > --- > drivers/net/virtio_net.c | 16 ++++++++++++---- > 1 file changed, 12 insertions(+), 4 deletions(-) > > diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c > index c29f42d1e04f..a83dc038d8af 100644 > --- a/drivers/net/virtio_net.c > +++ b/drivers/net/virtio_net.c > @@ -1433,7 +1433,10 @@ static void virtnet_poll_cleantx(struct receive_queue *rq) > return; > > if (__netif_tx_trylock(txq)) { > - free_old_xmit_skbs(sq, true); > + do { > + virtqueue_disable_cb(sq->vq); > + free_old_xmit_skbs(sq, true); > + } while (unlikely(!virtqueue_enable_cb_delayed(sq->vq))); > > if (sq->vq->num_free >= 2 + MAX_SKB_FRAGS) > netif_tx_wake_queue(txq); > @@ -1605,12 +1608,17 @@ static netdev_tx_t start_xmit(struct sk_buff *skb, struct net_device *dev) > struct netdev_queue *txq = netdev_get_tx_queue(dev, qnum); > bool kick = !netdev_xmit_more(); > bool use_napi = sq->napi.weight; > + unsigned int bytes = skb->len; > > /* Free up any pending old buffers before queueing new ones. */ > - free_old_xmit_skbs(sq, false); > + do { > + if (use_napi) > + virtqueue_disable_cb(sq->vq); > > - if (use_napi && kick) > - virtqueue_enable_cb_delayed(sq->vq); > + free_old_xmit_skbs(sq, false); > + > + } while (use_napi && kick && > + unlikely(!virtqueue_enable_cb_delayed(sq->vq))); > > /* timestamp packet in software */ > skb_tx_timestamp(skb); >
WARNING: multiple messages have this Message-ID (diff)
From: Eric Dumazet <eric.dumazet@gmail.com> To: "Michael S. Tsirkin" <mst@redhat.com>, linux-kernel@vger.kernel.org Cc: Willem de Bruijn <willemb@google.com>, netdev@vger.kernel.org, virtualization@lists.linux-foundation.org, Jakub Kicinski <kuba@kernel.org>, Wei Wang <weiwan@google.com>, David Miller <davem@davemloft.net> Subject: Re: [PATCH v3 4/4] virtio_net: disable cb aggressively Date: Wed, 26 May 2021 17:15:08 +0200 [thread overview] Message-ID: <18b47c22-8c8a-7699-ffaf-ccfdcbf39d37@gmail.com> (raw) In-Reply-To: <20210526082423.47837-5-mst@redhat.com> On 5/26/21 10:24 AM, Michael S. Tsirkin wrote: > There are currently two cases where we poll TX vq not in response to a > callback: start xmit and rx napi. We currently do this with callbacks > enabled which can cause extra interrupts from the card. Used not to be > a big issue as we run with interrupts disabled but that is no longer the > case, and in some cases the rate of spurious interrupts is so high > linux detects this and actually kills the interrupt. > > Fix up by disabling the callbacks before polling the tx vq. It is not clear why we want to poll TX completions from ndo_start_xmit() in napi mode ? This seems not needed, adding costs to sender thread, this might reduce the ability to use a different cpu for tx completions. Also this will likely conflict with BQL model if we want to use BQL at some point. > This probably needs a Fixes: tag > Signed-off-by: Michael S. Tsirkin <mst@redhat.com> > --- > drivers/net/virtio_net.c | 16 ++++++++++++---- > 1 file changed, 12 insertions(+), 4 deletions(-) > > diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c > index c29f42d1e04f..a83dc038d8af 100644 > --- a/drivers/net/virtio_net.c > +++ b/drivers/net/virtio_net.c > @@ -1433,7 +1433,10 @@ static void virtnet_poll_cleantx(struct receive_queue *rq) > return; > > if (__netif_tx_trylock(txq)) { > - free_old_xmit_skbs(sq, true); > + do { > + virtqueue_disable_cb(sq->vq); > + free_old_xmit_skbs(sq, true); > + } while (unlikely(!virtqueue_enable_cb_delayed(sq->vq))); > > if (sq->vq->num_free >= 2 + MAX_SKB_FRAGS) > netif_tx_wake_queue(txq); > @@ -1605,12 +1608,17 @@ static netdev_tx_t start_xmit(struct sk_buff *skb, struct net_device *dev) > struct netdev_queue *txq = netdev_get_tx_queue(dev, qnum); > bool kick = !netdev_xmit_more(); > bool use_napi = sq->napi.weight; > + unsigned int bytes = skb->len; > > /* Free up any pending old buffers before queueing new ones. */ > - free_old_xmit_skbs(sq, false); > + do { > + if (use_napi) > + virtqueue_disable_cb(sq->vq); > > - if (use_napi && kick) > - virtqueue_enable_cb_delayed(sq->vq); > + free_old_xmit_skbs(sq, false); > + > + } while (use_napi && kick && > + unlikely(!virtqueue_enable_cb_delayed(sq->vq))); > > /* timestamp packet in software */ > skb_tx_timestamp(skb); > _______________________________________________ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
next prev parent reply other threads:[~2021-05-26 15:15 UTC|newest] Thread overview: 49+ messages / expand[flat|nested] mbox.gz Atom feed top 2021-05-26 8:24 [PATCH v3 0/4] virtio net: spurious interrupt related fixes Michael S. Tsirkin 2021-05-26 8:24 ` Michael S. Tsirkin 2021-05-26 8:24 ` [PATCH v3 1/4] virtio_net: move tx vq operation under tx queue lock Michael S. Tsirkin 2021-05-26 8:24 ` Michael S. Tsirkin 2021-05-27 3:41 ` Jason Wang 2021-05-27 3:41 ` Jason Wang 2021-05-28 22:25 ` Willem de Bruijn 2021-05-28 22:25 ` Willem de Bruijn 2021-06-09 22:03 ` Michael S. Tsirkin 2021-06-09 22:03 ` Michael S. Tsirkin 2021-05-26 8:24 ` [PATCH v3 2/4] virtio_net: move txq wakeups under tx q lock Michael S. Tsirkin 2021-05-26 8:24 ` Michael S. Tsirkin 2021-05-27 3:48 ` Jason Wang 2021-05-27 3:48 ` Jason Wang 2021-05-26 8:24 ` [PATCH v3 3/4] virtio: fix up virtio_disable_cb Michael S. Tsirkin 2021-05-26 8:24 ` Michael S. Tsirkin 2021-05-27 4:01 ` Jason Wang 2021-05-27 4:01 ` Jason Wang 2023-03-30 6:07 ` Xuan Zhuo 2023-03-30 6:07 ` Xuan Zhuo 2023-03-30 6:44 ` Michael S. Tsirkin 2023-03-30 6:44 ` Michael S. Tsirkin 2023-03-30 6:54 ` Xuan Zhuo 2023-03-30 6:54 ` Xuan Zhuo 2023-03-30 14:04 ` Michael S. Tsirkin 2023-03-30 14:04 ` Michael S. Tsirkin 2023-03-31 3:38 ` Xuan Zhuo 2023-03-31 3:38 ` Xuan Zhuo 2021-05-26 8:24 ` [PATCH v3 4/4] virtio_net: disable cb aggressively Michael S. Tsirkin 2021-05-26 8:24 ` Michael S. Tsirkin 2021-05-26 15:15 ` Eric Dumazet [this message] 2021-05-26 15:15 ` Eric Dumazet 2021-05-26 21:22 ` Willem de Bruijn 2021-05-26 21:22 ` Willem de Bruijn 2021-05-26 19:39 ` Jakub Kicinski 2021-05-27 4:09 ` Jason Wang 2021-05-27 4:09 ` Jason Wang 2023-01-16 13:41 ` Laurent Vivier 2023-01-16 13:41 ` Laurent Vivier 2023-01-17 3:48 ` Jason Wang 2023-01-17 3:48 ` Jason Wang 2021-05-26 15:34 ` [PATCH v3 0/4] virtio net: spurious interrupt related fixes Willem de Bruijn 2021-05-26 15:34 ` Willem de Bruijn 2021-06-01 2:53 ` Willem de Bruijn 2021-06-01 2:53 ` Willem de Bruijn 2021-06-09 21:36 ` Willem de Bruijn 2021-06-09 21:36 ` Willem de Bruijn 2021-06-09 22:59 ` Willem de Bruijn 2021-06-09 22:59 ` Willem de Bruijn
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=18b47c22-8c8a-7699-ffaf-ccfdcbf39d37@gmail.com \ --to=eric.dumazet@gmail.com \ --cc=davem@davemloft.net \ --cc=jasowang@redhat.com \ --cc=kuba@kernel.org \ --cc=linux-kernel@vger.kernel.org \ --cc=mst@redhat.com \ --cc=netdev@vger.kernel.org \ --cc=virtualization@lists.linux-foundation.org \ --cc=weiwan@google.com \ --cc=willemb@google.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.