Netdev Archive on lore.kernel.org
 help / color / Atom feed
From: Yongji Xie <xieyongji@bytedance.com>
To: Parav Pandit <parav@nvidia.com>
Cc: Jason Wang <jasowang@redhat.com>,
	"virtualization@lists.linux-foundation.org" 
	<virtualization@lists.linux-foundation.org>,
	"Michael S. Tsirkin" <mst@redhat.com>,
	Eli Cohen <elic@nvidia.com>,
	"netdev@vger.kernel.org" <netdev@vger.kernel.org>
Subject: Re: [External] Re: [PATCH 0/7] Introduce vdpa management tool
Date: Wed, 2 Dec 2020 11:29:33 +0800
Message-ID: <CACycT3vNXvNVUP+oqzv-MMgtzneeeZUoMaDVtEws7VizH0V+mA@mail.gmail.com> (raw)
In-Reply-To: <BY5PR12MB4322C80FB3C76B85A2D095CCDCF40@BY5PR12MB4322.namprd12.prod.outlook.com>

On Tue, Dec 1, 2020 at 11:59 PM Parav Pandit <parav@nvidia.com> wrote:
>
>
>
> > From: Yongji Xie <xieyongji@bytedance.com>
> > Sent: Tuesday, December 1, 2020 7:49 PM
> >
> > On Tue, Dec 1, 2020 at 7:32 PM Parav Pandit <parav@nvidia.com> wrote:
> > >
> > >
> > >
> > > > From: Yongji Xie <xieyongji@bytedance.com>
> > > > Sent: Tuesday, December 1, 2020 3:26 PM
> > > >
> > > > On Tue, Dec 1, 2020 at 2:25 PM Jason Wang <jasowang@redhat.com>
> > wrote:
> > > > >
> > > > >
> > > > > On 2020/11/30 下午3:07, Yongji Xie wrote:
> > > > > >>> Thanks for adding me, Jason!
> > > > > >>>
> > > > > >>> Now I'm working on a v2 patchset for VDUSE (vDPA Device in
> > > > > >>> Userspace) [1]. This tool is very useful for the vduse device.
> > > > > >>> So I'm considering integrating this into my v2 patchset. But
> > > > > >>> there is one problem:
> > > > > >>>
> > > > > >>> In this tool, vdpa device config action and enable action are
> > > > > >>> combined into one netlink msg: VDPA_CMD_DEV_NEW. But in
> > vduse
> > > > > >>> case, it needs to be splitted because a chardev should be
> > > > > >>> created and opened by a userspace process before we enable the
> > > > > >>> vdpa device (call vdpa_register_device()).
> > > > > >>>
> > > > > >>> So I'd like to know whether it's possible (or have some plans)
> > > > > >>> to add two new netlink msgs something like:
> > > > > >>> VDPA_CMD_DEV_ENABLE
> > > > and
> > > > > >>> VDPA_CMD_DEV_DISABLE to make the config path more flexible.
> > > > > >>>
> > > > > >> Actually, we've discussed such intermediate step in some early
> > > > > >> discussion. It looks to me VDUSE could be one of the users of this.
> > > > > >>
> > > > > >> Or I wonder whether we can switch to use anonymous inode(fd)
> > > > > >> for VDUSE then fetching it via an VDUSE_GET_DEVICE_FD ioctl?
> > > > > >>
> > > > > > Yes, we can. Actually the current implementation in VDUSE is
> > > > > > like this.  But seems like this is still a intermediate step.
> > > > > > The fd should be binded to a name or something else which need
> > > > > > to be configured before.
> > > > >
> > > > >
> > > > > The name could be specified via the netlink. It looks to me the
> > > > > real issue is that until the device is connected with a userspace,
> > > > > it can't be used. So we also need to fail the enabling if it doesn't
> > opened.
> > > > >
> > > >
> > > > Yes, that's true. So you mean we can firstly try to fetch the fd
> > > > binded to a name/vduse_id via an VDUSE_GET_DEVICE_FD, then use the
> > > > name/vduse_id as a attribute to create vdpa device? It looks fine to me.
> > >
> > > I probably do not well understand. I tried reading patch [1] and few things
> > do not look correct as below.
> > > Creating the vdpa device on the bus device and destroying the device from
> > the workqueue seems unnecessary and racy.
> > >
> > > It seems vduse driver needs
> > > This is something should be done as part of the vdpa dev add command,
> > instead of connecting two sides separately and ensuring race free access to
> > it.
> > >
> > > So VDUSE_DEV_START and VDUSE_DEV_STOP should possibly be avoided.
> > >
> >
> > Yes, we can avoid these two ioctls with the help of the management tool.
> >
> > > $ vdpa dev add parentdev vduse_mgmtdev type net name foo2
> > >
> > > When above command is executed it creates necessary vdpa device foo2
> > on the bus.
> > > When user binds foo2 device with the vduse driver, in the probe(), it
> > creates respective char device to access it from user space.
> >
> I see. So vduse cannot work with any existing vdpa devices like ifc, mlx5 or netdevsim.
> It has its own implementation similar to fuse with its own backend of choice.
> More below.
>
> > But vduse driver is not a vdpa bus driver. It works like vdpasim driver, but
> > offloads the data plane and control plane to a user space process.
>
> In that case to draw parallel lines,
>
> 1. netdevsim:
> (a) create resources in kernel sw
> (b) datapath simulates in kernel
>
> 2. ifc + mlx5 vdpa dev:
> (a) creates resource in hw
> (b) data path is in hw
>
> 3. vduse:
> (a) creates resources in userspace sw
> (b) data path is in user space.
> hence creates data path resources for user space.
> So char device is created, removed as result of vdpa device creation.
>
> For example,
> $ vdpa dev add parentdev vduse_mgmtdev type net name foo2
>
> Above command will create char device for user space.
>
> Similar command for ifc/mlx5 would have created similar channel for rest of the config commands in hw.
> vduse channel = char device, eventfd etc.
> ifc/mlx5 hw channel = bar, irq, command interface etc
> Netdev sim channel = sw direct calls
>
> Does it make sense?

In my understanding, to make vdpa work, we need a backend (datapath
resources) and a frontend (a vdpa device attached to a vdpa bus). In
the above example, it looks like we use the command "vdpa dev add ..."
 to create a backend, so do we need another command to create a
frontend?

Thanks,
Yongji

  reply index

Thread overview: 79+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-11-12  6:39 Parav Pandit
2020-11-12  6:39 ` [PATCH 1/7] vdpa: Add missing comment for virtqueue count Parav Pandit
2020-11-12  6:40 ` [PATCH 2/7] vdpa: Use simpler version of ida allocation Parav Pandit
2020-11-12  6:40 ` [PATCH 3/7] vdpa: Extend routine to accept vdpa device name Parav Pandit
2020-11-12  6:40 ` [PATCH 4/7] vdpa: Define vdpa parent device, ops and a netlink interface Parav Pandit
2020-11-12  6:40 ` [PATCH 5/7] vdpa: Enable a user to add and delete a vdpa device Parav Pandit
2020-11-12  6:40 ` [PATCH 6/7] vdpa: Enable user to query vdpa device info Parav Pandit
2020-11-12  6:40 ` [PATCH 7/7] vdpa/vdpa_sim: Enable user to create vdpasim net devices Parav Pandit
2020-11-16  9:41 ` [PATCH 0/7] Introduce vdpa management tool Stefan Hajnoczi
2020-11-17 19:41   ` Parav Pandit
2020-11-16 22:23 ` Jakub Kicinski
2020-11-17 19:51   ` Parav Pandit
2020-12-16  9:13     ` Michael S. Tsirkin
     [not found]       ` <20201216080610.08541f44@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com>
2020-12-16 16:54         ` Parav Pandit
2020-12-16 19:57           ` Michael S. Tsirkin
2020-12-17 12:13             ` Parav Pandit
2020-11-27  3:53 ` Jason Wang
     [not found]   ` <CACycT3sYScObb9nN3g7L3cesjE7sCZWxZ5_5R1usGU9ePZEeqA@mail.gmail.com>
2020-11-30  3:36     ` [External] " Jason Wang
2020-11-30  7:07       ` Yongji Xie
2020-12-01  6:25         ` Jason Wang
2020-12-01  9:55           ` Yongji Xie
2020-12-01 11:32             ` Parav Pandit
2020-12-01 14:18               ` Yongji Xie
2020-12-01 15:58                 ` Parav Pandit
2020-12-02  3:29                   ` Yongji Xie [this message]
2020-12-02  4:53                     ` Parav Pandit
2020-12-02  5:51                       ` Jason Wang
2020-12-02  6:24                         ` Parav Pandit
2020-12-02  7:55                           ` Jason Wang
2020-12-02  9:27                         ` Yongji Xie
2020-12-02  9:21                       ` Yongji Xie
2020-12-02 11:13                         ` Parav Pandit
2020-12-02 13:18                           ` Yongji Xie
2020-12-02  5:48             ` Jason Wang
2020-12-08 22:47   ` David Ahern
2021-01-19  4:21     ` Parav Pandit
2020-12-16  9:16 ` Michael S. Tsirkin
2021-01-04  3:31 ` [PATCH linux-next v2 " Parav Pandit
2021-01-04  3:31   ` [PATCH linux-next v2 1/7] vdpa_sim_net: Make mac address array static Parav Pandit
2021-01-04  7:00     ` Jason Wang
2021-01-04  3:31   ` [PATCH linux-next v2 2/7] vdpa_sim_net: Add module param to disable default vdpa net device Parav Pandit
2021-01-04  3:31   ` [PATCH linux-next v2 3/7] vdpa: Extend routine to accept vdpa device name Parav Pandit
2021-01-04  3:31   ` [PATCH linux-next v2 4/7] vdpa: Define vdpa mgmt device, ops and a netlink interface Parav Pandit
2021-01-04  7:03     ` Jason Wang
2021-01-04  7:24       ` Parav Pandit
2021-01-05  4:10         ` Jason Wang
2021-01-05  6:33           ` Parav Pandit
2021-01-05  8:36             ` Jason Wang
2021-01-04  3:31   ` [PATCH linux-next v2 5/7] vdpa: Enable a user to add and delete a vdpa device Parav Pandit
2021-01-04  3:31   ` [PATCH linux-next v2 6/7] vdpa: Enable user to query vdpa device info Parav Pandit
2021-01-04  3:31   ` [PATCH linux-next v2 7/7] vdpa_sim_net: Add support for user supported devices Parav Pandit
2021-01-04  7:05     ` Jason Wang
2021-01-04  7:21       ` Parav Pandit
2021-01-05  4:06         ` Jason Wang
2021-01-05  6:22           ` Parav Pandit
2021-01-05 10:31 ` [PATCH linux-next v3 0/6] Introduce vdpa management tool Parav Pandit
2021-01-05 10:31   ` [PATCH linux-next v3 1/6] vdpa_sim_net: Make mac address array static Parav Pandit
2021-01-07 13:45     ` Stefano Garzarella
2021-01-05 10:31   ` [PATCH linux-next v3 2/6] vdpa: Extend routine to accept vdpa device name Parav Pandit
2021-01-05 10:32   ` [PATCH linux-next v3 3/6] vdpa: Define vdpa mgmt device, ops and a netlink interface Parav Pandit
2021-01-05 10:32   ` [PATCH linux-next v3 4/6] vdpa: Enable a user to add and delete a vdpa device Parav Pandit
2021-01-05 10:32   ` [PATCH linux-next v3 5/6] vdpa: Enable user to query vdpa device info Parav Pandit
2021-01-05 10:32   ` [PATCH linux-next v3 6/6] vdpa_sim_net: Add support for user supported devices Parav Pandit
2021-01-05 11:48     ` Michael S. Tsirkin
2021-01-05 12:02       ` Parav Pandit
2021-01-05 12:14         ` Michael S. Tsirkin
2021-01-05 12:30           ` Parav Pandit
2021-01-05 13:23             ` Michael S. Tsirkin
2021-01-07  3:48               ` Parav Pandit
2021-01-12  4:14                 ` Parav Pandit
2021-01-14  4:17                 ` Jason Wang
2021-01-14  7:58                   ` Parav Pandit
2021-01-15  5:38                     ` Jason Wang
2021-01-15  6:27                       ` Parav Pandit
2021-01-19 11:09                         ` Jason Wang
2021-01-20  3:21                           ` Parav Pandit
2021-01-20  3:46                             ` Parav Pandit
2021-01-18 18:03                       ` Parav Pandit
2021-01-20  7:53                         ` Michael S. Tsirkin

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=CACycT3vNXvNVUP+oqzv-MMgtzneeeZUoMaDVtEws7VizH0V+mA@mail.gmail.com \
    --to=xieyongji@bytedance.com \
    --cc=elic@nvidia.com \
    --cc=jasowang@redhat.com \
    --cc=mst@redhat.com \
    --cc=netdev@vger.kernel.org \
    --cc=parav@nvidia.com \
    --cc=virtualization@lists.linux-foundation.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Netdev Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/netdev/0 netdev/git/0.git
	git clone --mirror https://lore.kernel.org/netdev/1 netdev/git/1.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 netdev netdev/ https://lore.kernel.org/netdev \
		netdev@vger.kernel.org
	public-inbox-index netdev

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.netdev


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git