From mboxrd@z Thu Jan 1 00:00:00 1970 From: Willem de Bruijn Subject: Re: [PATCH net-next] virtio_net: ethtool tx napi configuration Date: Mon, 10 Sep 2018 09:35:27 -0400 Message-ID: References: <20180909224449.203593-1-willemdebruijn.kernel@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Cc: Network Development , David Miller , caleb.raitto@gmail.com, "Michael S. Tsirkin" , "Jon Olson (Google Drive)" , Willem de Bruijn To: Jason Wang Return-path: Received: from mail-ed1-f68.google.com ([209.85.208.68]:39680 "EHLO mail-ed1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727970AbeIJSaO (ORCPT ); Mon, 10 Sep 2018 14:30:14 -0400 Received: by mail-ed1-f68.google.com with SMTP id h4-v6so16522945edi.6 for ; Mon, 10 Sep 2018 06:36:05 -0700 (PDT) In-Reply-To: Sender: netdev-owner@vger.kernel.org List-ID: On Mon, Sep 10, 2018 at 2:01 AM Jason Wang wrote: > > > > On 2018=E5=B9=B409=E6=9C=8810=E6=97=A5 06:44, Willem de Bruijn wrote: > > From: Willem de Bruijn > > > > Implement ethtool .set_coalesce (-C) and .get_coalesce (-c) handlers. > > Interrupt moderation is currently not supported, so these accept and > > display the default settings of 0 usec and 1 frame. > > > > Toggle tx napi through a bit in tx-frames. So as to not interfere > > with possible future interrupt moderation, use bit 10, well outside > > the reasonable range of real interrupt moderation values. > > > > Changes are not atomic. The tx IRQ, napi BH and transmit path must > > be quiesced when switching modes. Only allow changing this setting > > when the device is down. > > I cook a fixup, and it looks works in my setup: > > diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c > index b320b6b14749..9181c3f2f832 100644 > --- a/drivers/net/virtio_net.c > +++ b/drivers/net/virtio_net.c > @@ -2204,10 +2204,17 @@ static int virtnet_set_coalesce(struct > net_device *dev, > return -EINVAL; > > if (napi_weight ^ vi->sq[0].napi.weight) { > - if (dev->flags & IFF_UP) > - return -EBUSY; > - for (i =3D 0; i < vi->max_queue_pairs; i++) > + for (i =3D 0; i < vi->max_queue_pairs; i++) { > + struct netdev_queue *txq =3D > + netdev_get_tx_queue(vi->dev, i); > + > + virtnet_napi_tx_disable(&vi->sq[i].napi); > + __netif_tx_lock_bh(txq); > vi->sq[i].napi.weight =3D napi_weight; > + __netif_tx_unlock_bh(txq); > + virtnet_napi_tx_enable(vi, vi->sq[i].vq, > + &vi->sq[i].napi); > + } > } > > return 0; Thanks! It passes my simple stress test, too. Which consists of two concurrent loops, one toggling the ethtool option, another running TCP_RR. > The only left case is the speculative tx polling in RX NAPI. I think we > don't need to care in this case since it was not a must for correctness. As long as the txq lock is held that will be a noop, anyway. The other concurrent action is skb_xmit_done. It looks correct to me, but need to think about it a bit. The tricky transition is coming out of napi withou= t having >=3D 2 + MAX_SKB_FRAGS clean descriptors. If the queue is stopped it may deadlock transmission in no-napi mode. > > > > Link: https://patchwork.ozlabs.org/patch/948149/ > > Suggested-by: Jason Wang > > Signed-off-by: Willem de Bruijn > > --- > > drivers/net/virtio_net.c | 52 +++++++++++++++++++++++++++++++++++++++= + > > 1 file changed, 52 insertions(+) > > > > diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c > > index 765920905226..b320b6b14749 100644 > > --- a/drivers/net/virtio_net.c > > +++ b/drivers/net/virtio_net.c > > @@ -66,6 +66,8 @@ DECLARE_EWMA(pkt_len, 0, 64) > > > > #define VIRTNET_DRIVER_VERSION "1.0.0" > > > > +static const u32 ethtool_coalesce_napi_mask =3D (1UL << 10); > > + > > static const unsigned long guest_offloads[] =3D { > > VIRTIO_NET_F_GUEST_TSO4, > > VIRTIO_NET_F_GUEST_TSO6, > > @@ -2181,6 +2183,54 @@ static int virtnet_get_link_ksettings(struct net= _device *dev, > > return 0; > > } > > > > +static int virtnet_set_coalesce(struct net_device *dev, > > + struct ethtool_coalesce *ec) > > +{ > > + const struct ethtool_coalesce ec_default =3D { > > + .cmd =3D ETHTOOL_SCOALESCE, > > + .rx_max_coalesced_frames =3D 1, > > I think rx part is no necessary. The definition of ethtool_coalesce has: "* It is illegal to set both usecs and max_frames to zero as this * would cause interrupts to never be generated. To disable * coalescing, set usecs =3D 0 and max_frames =3D 1." I'd rather not diverge from this prescribed behavior unless there's a strong reason. On the related point in the other thread: > Rethink about this, how about something like: > > - UINT_MAX: no tx interrupt > - other value: tx interrupt with possible interrupt moderation Okay, that will be simpler to configure.