[RFC] Enable primary/secondary model for vdev

[RFC] Enable primary/secondary model for vdev

[Tan, Jianfeng]

Hi,

Status quo:
Almost none of vdev supports primary/secondary model. Two exceptions are 
rte_ring, virtio-user (limited support). Two problems facing this issue.

P1: How to attach vdev in secondary process?

Previous discussion: http://dpdk.org/ml/archives/dev/2017-April/063370.html
According current implementation, vdev can be recognized on vdev bus 
only if it is explicitly assigned in the parameters, or invokes call 
rte_eal_vdev_init() to create one. Assigning --vdev parameter explicitly 
on secondary process is not the optimal way. Instead, we can iterate 
rte_eth_dev_data array to discover/obtain all devices, which seems a 
more unified solution with PCI devices.

If we only need to obtain the statistics of vdev in secondary process 
(like what dpdk-procinfo does), solving above issue is enough, but if we 
need to Rx/Tx packets in secondary process for vdev, we need to solve 
the following problem too.

P2: How to share dev-specific FDs to secondary process?

We need to share the FDs which are opened at primary process, for 
example, the socket file of pcap, the char device of tap, the QEMU 
memory region fds of vhost-user, etc. The only (as far as we know) way 
to share FD between processes in Linux is to use ancillary data of 
sendmsg/recvmsg on unix socket. That means we propose a new socket file 
created for each primary proess, and secondary processes will connect 
with this unix socket; and for simplicity, we also propose a new API to 
share FD, so that vdev can make use of it to share FDs.

Another problem I'd like to make clear is: can secondary processes own 
ports that does not belong to primary?

Besides, there are two errors in documents: (1) rte_ring supports 
primary/secondary model but the document does not state that; (2) pcap 
does not support primary/secondary model but the document states it 
supports.

Thanks,
Jianfeng

Re: [RFC] Enable primary/secondary model for vdev

[Tan, Jianfeng]

Any comments please?

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Tan, Jianfeng
> Sent: Thursday, May 11, 2017 2:20 PM
> To: dev@dpdk.org
> Cc: Thomas Monjalon; Richardson, Bruce; Ananyev, Konstantin; Stephen
> Hemminger; Yuanhan Liu; Yigit, Ferruh; Yang, Zhiyong;
> huanghuai@meituan.com
> Subject: [dpdk-dev] [RFC] Enable primary/secondary model for vdev
> 
> Hi,
> 
> Status quo:
> Almost none of vdev supports primary/secondary model. Two exceptions
> are
> rte_ring, virtio-user (limited support). Two problems facing this issue.
> 
> P1: How to attach vdev in secondary process?
> 
> Previous discussion: http://dpdk.org/ml/archives/dev/2017-
> April/063370.html
> According current implementation, vdev can be recognized on vdev bus
> only if it is explicitly assigned in the parameters, or invokes call
> rte_eal_vdev_init() to create one. Assigning --vdev parameter explicitly
> on secondary process is not the optimal way. Instead, we can iterate
> rte_eth_dev_data array to discover/obtain all devices, which seems a
> more unified solution with PCI devices.
> 
> If we only need to obtain the statistics of vdev in secondary process
> (like what dpdk-procinfo does), solving above issue is enough, but if we
> need to Rx/Tx packets in secondary process for vdev, we need to solve
> the following problem too.
> 
> P2: How to share dev-specific FDs to secondary process?
> 
> We need to share the FDs which are opened at primary process, for
> example, the socket file of pcap, the char device of tap, the QEMU
> memory region fds of vhost-user, etc. The only (as far as we know) way
> to share FD between processes in Linux is to use ancillary data of
> sendmsg/recvmsg on unix socket. That means we propose a new socket file
> created for each primary proess, and secondary processes will connect
> with this unix socket; and for simplicity, we also propose a new API to
> share FD, so that vdev can make use of it to share FDs.
> 
> Another problem I'd like to make clear is: can secondary processes own
> ports that does not belong to primary?
> 
> Besides, there are two errors in documents: (1) rte_ring supports
> primary/secondary model but the document does not state that; (2) pcap
> does not support primary/secondary model but the document states it
> supports.
> 
> Thanks,
> Jianfeng