All of lore.kernel.org
 help / color / mirror / Atom feed
From: Magnus Karlsson <magnus.karlsson@gmail.com>
To: Jason Wang <jasowang@redhat.com>
Cc: "Xuan Zhuo" <xuanzhuo@linux.alibaba.com>,
	"Network Development" <netdev@vger.kernel.org>,
	"Michael S. Tsirkin" <mst@redhat.com>,
	"David S. Miller" <davem@davemloft.net>,
	"Jakub Kicinski" <kuba@kernel.org>,
	"Björn Töpel" <bjorn@kernel.org>,
	"Magnus Karlsson" <magnus.karlsson@intel.com>,
	"Jonathan Lemon" <jonathan.lemon@gmail.com>,
	"Alexei Starovoitov" <ast@kernel.org>,
	"Daniel Borkmann" <daniel@iogearbox.net>,
	"Jesper Dangaard Brouer" <hawk@kernel.org>,
	"John Fastabend" <john.fastabend@gmail.com>,
	"Andrii Nakryiko" <andrii@kernel.org>,
	"Martin KaFai Lau" <kafai@fb.com>,
	"Song Liu" <songliubraving@fb.com>, "Yonghong Song" <yhs@fb.com>,
	"KP Singh" <kpsingh@kernel.org>,
	virtualization@lists.linux-foundation.org,
	bpf <bpf@vger.kernel.org>, "Dust Li" <dust.li@linux.alibaba.com>
Subject: Re: [PATCH net-next v3 3/8] virtio-net: xsk zero copy xmit setup
Date: Wed, 7 Apr 2021 12:02:37 +0200	[thread overview]
Message-ID: <CAJ8uoz3Li9XrFQ=WH5u22EBFkK97RKqDcGGoX6W963w=RsQ9Pw@mail.gmail.com> (raw)
In-Reply-To: <97a147bc-f9b8-ce95-889d-274893fd0444@redhat.com>

On Tue, Apr 6, 2021 at 3:13 PM Jason Wang <jasowang@redhat.com> wrote:
>
>
> 在 2021/3/31 下午3:11, Xuan Zhuo 写道:
> > xsk is a high-performance packet receiving and sending technology.
> >
> > This patch implements the binding and unbinding operations of xsk and
> > the virtio-net queue for xsk zero copy xmit.
> >
> > The xsk zero copy xmit depends on tx napi. So if tx napi is not opened,
> > an error will be reported. And the entire operation is under the
> > protection of rtnl_lock
> >
> > Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> > Reviewed-by: Dust Li <dust.li@linux.alibaba.com>
> > ---
> >   drivers/net/virtio_net.c | 66 ++++++++++++++++++++++++++++++++++++++++
> >   1 file changed, 66 insertions(+)
> >
> > diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
> > index bb4ea9dbc16b..4e25408a2b37 100644
> > --- a/drivers/net/virtio_net.c
> > +++ b/drivers/net/virtio_net.c
> > @@ -22,6 +22,7 @@
> >   #include <net/route.h>
> >   #include <net/xdp.h>
> >   #include <net/net_failover.h>
> > +#include <net/xdp_sock_drv.h>
> >
> >   static int napi_weight = NAPI_POLL_WEIGHT;
> >   module_param(napi_weight, int, 0444);
> > @@ -133,6 +134,11 @@ struct send_queue {
> >       struct virtnet_sq_stats stats;
> >
> >       struct napi_struct napi;
> > +
> > +     struct {
> > +             /* xsk pool */
> > +             struct xsk_buff_pool __rcu *pool;
> > +     } xsk;
> >   };
> >
> >   /* Internal representation of a receive virtqueue */
> > @@ -2526,11 +2532,71 @@ static int virtnet_xdp_set(struct net_device *dev, struct bpf_prog *prog,
> >       return err;
> >   }
> >
> > +static int virtnet_xsk_pool_enable(struct net_device *dev,
> > +                                struct xsk_buff_pool *pool,
> > +                                u16 qid)
> > +{
> > +     struct virtnet_info *vi = netdev_priv(dev);
> > +     struct send_queue *sq;
> > +     int ret = -EBUSY;
>
>
> I'd rather move this under the check of xsk.pool.
>
>
> > +
> > +     if (qid >= vi->curr_queue_pairs)
> > +             return -EINVAL;
> > +
> > +     sq = &vi->sq[qid];
> > +
> > +     /* xsk zerocopy depend on the tx napi */
>
>
> Need more comments to explain why tx NAPI is required here.
>
> And what's more important, tx NAPI could be enabled/disable via ethtool,
> what if the NAPI is disabled after xsk is enabled?
>
>
> > +     if (!sq->napi.weight)
> > +             return -EPERM;
> > +
> > +     rcu_read_lock();
> > +     if (rcu_dereference(sq->xsk.pool))
> > +             goto end;
>
>
> Under what condition can we reach here?

There is already code in the common xsk part that tests for binding
multiple sockets to the same netdev and queue id (in an illegal way
that is). Does this code not work for you? If so, we should fix that
and not introduce a separate check down in the driver. Or maybe I
misunderstand your problem.

The code is here:

struct xsk_buff_pool *xsk_get_pool_from_qid(struct net_device *dev,
                                            u16 queue_id)
{
        if (queue_id < dev->real_num_rx_queues)
                return dev->_rx[queue_id].pool;
        if (queue_id < dev->real_num_tx_queues)
                return dev->_tx[queue_id].pool;

        return NULL;
}

int xp_assign_dev(struct xsk_buff_pool *pool,
                  struct net_device *netdev, u16 queue_id, u16 flags)
{
        :
        :
        if (xsk_get_pool_from_qid(netdev, queue_id))
                return -EBUSY;


>
> > +
> > +     /* Here is already protected by rtnl_lock, so rcu_assign_pointer is
> > +      * safe.
> > +      */
> > +     rcu_assign_pointer(sq->xsk.pool, pool);
> > +     ret = 0;
> > +end:
> > +     rcu_read_unlock();
> > +
> > +     return ret;
> > +}
> > +
> > +static int virtnet_xsk_pool_disable(struct net_device *dev, u16 qid)
> > +{
> > +     struct virtnet_info *vi = netdev_priv(dev);
> > +     struct send_queue *sq;
> > +
> > +     if (qid >= vi->curr_queue_pairs)
> > +             return -EINVAL;
> > +
> > +     sq = &vi->sq[qid];
> > +
> > +     /* Here is already protected by rtnl_lock, so rcu_assign_pointer is
> > +      * safe.
> > +      */
> > +     rcu_assign_pointer(sq->xsk.pool, NULL);
> > +
> > +     synchronize_rcu(); /* Sync with the XSK wakeup and with NAPI. */
>
>
> Since rtnl is held here, I guess it's better to use synchornize_net().
>
>
> > +
> > +     return 0;
> > +}
> > +
> >   static int virtnet_xdp(struct net_device *dev, struct netdev_bpf *xdp)
> >   {
> >       switch (xdp->command) {
> >       case XDP_SETUP_PROG:
> >               return virtnet_xdp_set(dev, xdp->prog, xdp->extack);
> > +     case XDP_SETUP_XSK_POOL:
> > +             /* virtio net not use dma before call vring api */
> > +             xdp->xsk.check_dma = false;
>
>
> I think it's better not open code things like this. How about introduce
> new parameters in xp_assign_dev()?
>
>
> > +             if (xdp->xsk.pool)
> > +                     return virtnet_xsk_pool_enable(dev, xdp->xsk.pool,
> > +                                                    xdp->xsk.queue_id);
> > +             else
> > +                     return virtnet_xsk_pool_disable(dev, xdp->xsk.queue_id);
> >       default:
> >               return -EINVAL;
> >       }
>
>
> Thanks
>

  reply	other threads:[~2021-04-07 10:03 UTC|newest]

Thread overview: 27+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-03-31  7:11 [PATCH net-next v3 0/8] virtio-net support xdp socket zero copy xmit Xuan Zhuo
2021-03-31  7:11 ` [PATCH net-next v3 1/8] xsk: XDP_SETUP_XSK_POOL support option check_dma Xuan Zhuo
2021-03-31  7:11 ` [PATCH net-next v3 2/8] xsk: support get page by addr Xuan Zhuo
2021-03-31  7:11 ` [PATCH net-next v3 3/8] virtio-net: xsk zero copy xmit setup Xuan Zhuo
2021-04-06  4:27   ` Jason Wang
2021-04-06  4:27     ` Jason Wang
2021-04-07 10:02     ` Magnus Karlsson [this message]
2021-03-31  7:11 ` [PATCH net-next v3 4/8] virtio-net: xsk zero copy xmit implement wakeup and xmit Xuan Zhuo
2021-04-06  6:19   ` Jason Wang
2021-04-06  6:19     ` Jason Wang
2021-04-07  9:56     ` Magnus Karlsson
2021-03-31  7:11 ` [PATCH net-next v3 5/8] virtio-net: xsk zero copy xmit support xsk unaligned mode Xuan Zhuo
2021-04-06  6:55   ` Jason Wang
2021-04-06  6:55     ` Jason Wang
2021-03-31  7:11 ` [PATCH net-next v3 6/8] virtio-net: xsk zero copy xmit kick by threshold Xuan Zhuo
2021-04-06  6:59   ` Jason Wang
2021-04-06  6:59     ` Jason Wang
2021-03-31  7:11 ` [PATCH net-next v3 7/8] virtio-net: poll tx call xsk zerocopy xmit Xuan Zhuo
2021-04-06  7:03   ` Jason Wang
2021-04-06  7:03     ` Jason Wang
2021-03-31  7:11 ` [PATCH net-next v3 8/8] virtio-net: free old xmit handle xsk Xuan Zhuo
2021-03-31 10:09   ` kernel test robot
2021-03-31 10:09     ` kernel test robot
2021-04-06  7:16   ` Jason Wang
2021-04-06  7:16     ` Jason Wang
     [not found] <1617780476.5300975-1-xuanzhuo@linux.alibaba.com>
2021-04-07  9:00 ` [PATCH net-next v3 3/8] virtio-net: xsk zero copy xmit setup Jason Wang
2021-04-07  9:00   ` Jason Wang

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CAJ8uoz3Li9XrFQ=WH5u22EBFkK97RKqDcGGoX6W963w=RsQ9Pw@mail.gmail.com' \
    --to=magnus.karlsson@gmail.com \
    --cc=andrii@kernel.org \
    --cc=ast@kernel.org \
    --cc=bjorn@kernel.org \
    --cc=bpf@vger.kernel.org \
    --cc=daniel@iogearbox.net \
    --cc=davem@davemloft.net \
    --cc=dust.li@linux.alibaba.com \
    --cc=hawk@kernel.org \
    --cc=jasowang@redhat.com \
    --cc=john.fastabend@gmail.com \
    --cc=jonathan.lemon@gmail.com \
    --cc=kafai@fb.com \
    --cc=kpsingh@kernel.org \
    --cc=kuba@kernel.org \
    --cc=magnus.karlsson@intel.com \
    --cc=mst@redhat.com \
    --cc=netdev@vger.kernel.org \
    --cc=songliubraving@fb.com \
    --cc=virtualization@lists.linux-foundation.org \
    --cc=xuanzhuo@linux.alibaba.com \
    --cc=yhs@fb.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.