Netdev Archive on lore.kernel.org
 help / color / Atom feed
From: Parav Pandit <parav@nvidia.com>
To: Yongji Xie <xieyongji@bytedance.com>
Cc: Jason Wang <jasowang@redhat.com>,
	"virtualization@lists.linux-foundation.org" 
	<virtualization@lists.linux-foundation.org>,
	"Michael S. Tsirkin" <mst@redhat.com>,
	Eli Cohen <elic@nvidia.com>,
	"netdev@vger.kernel.org" <netdev@vger.kernel.org>
Subject: RE: [External] Re: [PATCH 0/7] Introduce vdpa management tool
Date: Tue, 1 Dec 2020 15:58:56 +0000
Message-ID: <BY5PR12MB4322C80FB3C76B85A2D095CCDCF40@BY5PR12MB4322.namprd12.prod.outlook.com> (raw)
In-Reply-To: <CACycT3tTCmEzY37E5196Q2mqME2v+KpAp7Snn8wK4XtRKHEqEw@mail.gmail.com>



> From: Yongji Xie <xieyongji@bytedance.com>
> Sent: Tuesday, December 1, 2020 7:49 PM
> 
> On Tue, Dec 1, 2020 at 7:32 PM Parav Pandit <parav@nvidia.com> wrote:
> >
> >
> >
> > > From: Yongji Xie <xieyongji@bytedance.com>
> > > Sent: Tuesday, December 1, 2020 3:26 PM
> > >
> > > On Tue, Dec 1, 2020 at 2:25 PM Jason Wang <jasowang@redhat.com>
> wrote:
> > > >
> > > >
> > > > On 2020/11/30 下午3:07, Yongji Xie wrote:
> > > > >>> Thanks for adding me, Jason!
> > > > >>>
> > > > >>> Now I'm working on a v2 patchset for VDUSE (vDPA Device in
> > > > >>> Userspace) [1]. This tool is very useful for the vduse device.
> > > > >>> So I'm considering integrating this into my v2 patchset. But
> > > > >>> there is one problem:
> > > > >>>
> > > > >>> In this tool, vdpa device config action and enable action are
> > > > >>> combined into one netlink msg: VDPA_CMD_DEV_NEW. But in
> vduse
> > > > >>> case, it needs to be splitted because a chardev should be
> > > > >>> created and opened by a userspace process before we enable the
> > > > >>> vdpa device (call vdpa_register_device()).
> > > > >>>
> > > > >>> So I'd like to know whether it's possible (or have some plans)
> > > > >>> to add two new netlink msgs something like:
> > > > >>> VDPA_CMD_DEV_ENABLE
> > > and
> > > > >>> VDPA_CMD_DEV_DISABLE to make the config path more flexible.
> > > > >>>
> > > > >> Actually, we've discussed such intermediate step in some early
> > > > >> discussion. It looks to me VDUSE could be one of the users of this.
> > > > >>
> > > > >> Or I wonder whether we can switch to use anonymous inode(fd)
> > > > >> for VDUSE then fetching it via an VDUSE_GET_DEVICE_FD ioctl?
> > > > >>
> > > > > Yes, we can. Actually the current implementation in VDUSE is
> > > > > like this.  But seems like this is still a intermediate step.
> > > > > The fd should be binded to a name or something else which need
> > > > > to be configured before.
> > > >
> > > >
> > > > The name could be specified via the netlink. It looks to me the
> > > > real issue is that until the device is connected with a userspace,
> > > > it can't be used. So we also need to fail the enabling if it doesn't
> opened.
> > > >
> > >
> > > Yes, that's true. So you mean we can firstly try to fetch the fd
> > > binded to a name/vduse_id via an VDUSE_GET_DEVICE_FD, then use the
> > > name/vduse_id as a attribute to create vdpa device? It looks fine to me.
> >
> > I probably do not well understand. I tried reading patch [1] and few things
> do not look correct as below.
> > Creating the vdpa device on the bus device and destroying the device from
> the workqueue seems unnecessary and racy.
> >
> > It seems vduse driver needs
> > This is something should be done as part of the vdpa dev add command,
> instead of connecting two sides separately and ensuring race free access to
> it.
> >
> > So VDUSE_DEV_START and VDUSE_DEV_STOP should possibly be avoided.
> >
> 
> Yes, we can avoid these two ioctls with the help of the management tool.
> 
> > $ vdpa dev add parentdev vduse_mgmtdev type net name foo2
> >
> > When above command is executed it creates necessary vdpa device foo2
> on the bus.
> > When user binds foo2 device with the vduse driver, in the probe(), it
> creates respective char device to access it from user space.
>
I see. So vduse cannot work with any existing vdpa devices like ifc, mlx5 or netdevsim.
It has its own implementation similar to fuse with its own backend of choice.
More below.

> But vduse driver is not a vdpa bus driver. It works like vdpasim driver, but
> offloads the data plane and control plane to a user space process.

In that case to draw parallel lines,

1. netdevsim:
(a) create resources in kernel sw
(b) datapath simulates in kernel

2. ifc + mlx5 vdpa dev:
(a) creates resource in hw
(b) data path is in hw

3. vduse:
(a) creates resources in userspace sw
(b) data path is in user space.
hence creates data path resources for user space.
So char device is created, removed as result of vdpa device creation.

For example,
$ vdpa dev add parentdev vduse_mgmtdev type net name foo2

Above command will create char device for user space.

Similar command for ifc/mlx5 would have created similar channel for rest of the config commands in hw.
vduse channel = char device, eventfd etc.
ifc/mlx5 hw channel = bar, irq, command interface etc
Netdev sim channel = sw direct calls

Does it make sense?

  reply index

Thread overview: 79+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-11-12  6:39 Parav Pandit
2020-11-12  6:39 ` [PATCH 1/7] vdpa: Add missing comment for virtqueue count Parav Pandit
2020-11-12  6:40 ` [PATCH 2/7] vdpa: Use simpler version of ida allocation Parav Pandit
2020-11-12  6:40 ` [PATCH 3/7] vdpa: Extend routine to accept vdpa device name Parav Pandit
2020-11-12  6:40 ` [PATCH 4/7] vdpa: Define vdpa parent device, ops and a netlink interface Parav Pandit
2020-11-12  6:40 ` [PATCH 5/7] vdpa: Enable a user to add and delete a vdpa device Parav Pandit
2020-11-12  6:40 ` [PATCH 6/7] vdpa: Enable user to query vdpa device info Parav Pandit
2020-11-12  6:40 ` [PATCH 7/7] vdpa/vdpa_sim: Enable user to create vdpasim net devices Parav Pandit
2020-11-16  9:41 ` [PATCH 0/7] Introduce vdpa management tool Stefan Hajnoczi
2020-11-17 19:41   ` Parav Pandit
2020-11-16 22:23 ` Jakub Kicinski
2020-11-17 19:51   ` Parav Pandit
2020-12-16  9:13     ` Michael S. Tsirkin
     [not found]       ` <20201216080610.08541f44@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com>
2020-12-16 16:54         ` Parav Pandit
2020-12-16 19:57           ` Michael S. Tsirkin
2020-12-17 12:13             ` Parav Pandit
2020-11-27  3:53 ` Jason Wang
     [not found]   ` <CACycT3sYScObb9nN3g7L3cesjE7sCZWxZ5_5R1usGU9ePZEeqA@mail.gmail.com>
2020-11-30  3:36     ` [External] " Jason Wang
2020-11-30  7:07       ` Yongji Xie
2020-12-01  6:25         ` Jason Wang
2020-12-01  9:55           ` Yongji Xie
2020-12-01 11:32             ` Parav Pandit
2020-12-01 14:18               ` Yongji Xie
2020-12-01 15:58                 ` Parav Pandit [this message]
2020-12-02  3:29                   ` Yongji Xie
2020-12-02  4:53                     ` Parav Pandit
2020-12-02  5:51                       ` Jason Wang
2020-12-02  6:24                         ` Parav Pandit
2020-12-02  7:55                           ` Jason Wang
2020-12-02  9:27                         ` Yongji Xie
2020-12-02  9:21                       ` Yongji Xie
2020-12-02 11:13                         ` Parav Pandit
2020-12-02 13:18                           ` Yongji Xie
2020-12-02  5:48             ` Jason Wang
2020-12-08 22:47   ` David Ahern
2021-01-19  4:21     ` Parav Pandit
2020-12-16  9:16 ` Michael S. Tsirkin
2021-01-04  3:31 ` [PATCH linux-next v2 " Parav Pandit
2021-01-04  3:31   ` [PATCH linux-next v2 1/7] vdpa_sim_net: Make mac address array static Parav Pandit
2021-01-04  7:00     ` Jason Wang
2021-01-04  3:31   ` [PATCH linux-next v2 2/7] vdpa_sim_net: Add module param to disable default vdpa net device Parav Pandit
2021-01-04  3:31   ` [PATCH linux-next v2 3/7] vdpa: Extend routine to accept vdpa device name Parav Pandit
2021-01-04  3:31   ` [PATCH linux-next v2 4/7] vdpa: Define vdpa mgmt device, ops and a netlink interface Parav Pandit
2021-01-04  7:03     ` Jason Wang
2021-01-04  7:24       ` Parav Pandit
2021-01-05  4:10         ` Jason Wang
2021-01-05  6:33           ` Parav Pandit
2021-01-05  8:36             ` Jason Wang
2021-01-04  3:31   ` [PATCH linux-next v2 5/7] vdpa: Enable a user to add and delete a vdpa device Parav Pandit
2021-01-04  3:31   ` [PATCH linux-next v2 6/7] vdpa: Enable user to query vdpa device info Parav Pandit
2021-01-04  3:31   ` [PATCH linux-next v2 7/7] vdpa_sim_net: Add support for user supported devices Parav Pandit
2021-01-04  7:05     ` Jason Wang
2021-01-04  7:21       ` Parav Pandit
2021-01-05  4:06         ` Jason Wang
2021-01-05  6:22           ` Parav Pandit
2021-01-05 10:31 ` [PATCH linux-next v3 0/6] Introduce vdpa management tool Parav Pandit
2021-01-05 10:31   ` [PATCH linux-next v3 1/6] vdpa_sim_net: Make mac address array static Parav Pandit
2021-01-07 13:45     ` Stefano Garzarella
2021-01-05 10:31   ` [PATCH linux-next v3 2/6] vdpa: Extend routine to accept vdpa device name Parav Pandit
2021-01-05 10:32   ` [PATCH linux-next v3 3/6] vdpa: Define vdpa mgmt device, ops and a netlink interface Parav Pandit
2021-01-05 10:32   ` [PATCH linux-next v3 4/6] vdpa: Enable a user to add and delete a vdpa device Parav Pandit
2021-01-05 10:32   ` [PATCH linux-next v3 5/6] vdpa: Enable user to query vdpa device info Parav Pandit
2021-01-05 10:32   ` [PATCH linux-next v3 6/6] vdpa_sim_net: Add support for user supported devices Parav Pandit
2021-01-05 11:48     ` Michael S. Tsirkin
2021-01-05 12:02       ` Parav Pandit
2021-01-05 12:14         ` Michael S. Tsirkin
2021-01-05 12:30           ` Parav Pandit
2021-01-05 13:23             ` Michael S. Tsirkin
2021-01-07  3:48               ` Parav Pandit
2021-01-12  4:14                 ` Parav Pandit
2021-01-14  4:17                 ` Jason Wang
2021-01-14  7:58                   ` Parav Pandit
2021-01-15  5:38                     ` Jason Wang
2021-01-15  6:27                       ` Parav Pandit
2021-01-19 11:09                         ` Jason Wang
2021-01-20  3:21                           ` Parav Pandit
2021-01-20  3:46                             ` Parav Pandit
2021-01-18 18:03                       ` Parav Pandit
2021-01-20  7:53                         ` Michael S. Tsirkin

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=BY5PR12MB4322C80FB3C76B85A2D095CCDCF40@BY5PR12MB4322.namprd12.prod.outlook.com \
    --to=parav@nvidia.com \
    --cc=elic@nvidia.com \
    --cc=jasowang@redhat.com \
    --cc=mst@redhat.com \
    --cc=netdev@vger.kernel.org \
    --cc=virtualization@lists.linux-foundation.org \
    --cc=xieyongji@bytedance.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Netdev Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/netdev/0 netdev/git/0.git
	git clone --mirror https://lore.kernel.org/netdev/1 netdev/git/1.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 netdev netdev/ https://lore.kernel.org/netdev \
		netdev@vger.kernel.org
	public-inbox-index netdev

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.netdev


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git