Re: [dpdk-dev] [PATCH v2 3/6] ethdev: add API to get hairpin peer ports list

[Ori Kam <orika@nvidia.com>]

Hi Bing,

PSB,
Best,
Ori
> -----Original Message-----
> From: Bing Zhao <bingz@nvidia.com>
> Sent: Thursday, October 8, 2020 11:52 AM
> Subject: [PATCH v2 3/6] ethdev: add API to get hairpin peer ports list
> 
> After hairpin queues are configured, in general, the application will
> maintain the ports topology and even the queues configuration for
> hairpin. But sometimes it will not.
> 
> If there is no hot-plug, it is easy to bind and unbind hairpin among
> all the ports. The application can just connect or disconnect the
> hairpin egress ports to / from all the probed ingress ports. Then
> all the connections could be handled properly.
> 
> But with hot-plug / hot-unplug, one port could be probed and removed
> dynamically. With two ports hairpin, all the connections from and to
> this port should be handled after start(bind) or before stop(unbind).
> It is necessary to know the hairpin topology with this port.
> 
> This API will return the ports list with the actual peer ports number
> after configuration. Either peer RX or TX ports will be gotten with
> this function call.
> 
> Signed-off-by: Bing Zhao <bingz@nvidia.com>
> ---
>  lib/librte_ethdev/rte_ethdev.c           | 24 ++++++++++++++++++++++++
>  lib/librte_ethdev/rte_ethdev.h           | 27 +++++++++++++++++++++++++++
>  lib/librte_ethdev/rte_ethdev_driver.h    | 30
> ++++++++++++++++++++++++++++++
>  lib/librte_ethdev/rte_ethdev_version.map |  1 +
>  4 files changed, 82 insertions(+)
> 
> diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
> index a4adeff..ea7892a 100644
> --- a/lib/librte_ethdev/rte_ethdev.c
> +++ b/lib/librte_ethdev/rte_ethdev.c
> @@ -2208,6 +2208,30 @@ struct rte_eth_dev *
>  	return ret;
>  }
> 
> +int
> +rte_eth_hairpin_get_peer_ports(uint16_t cur_port, uint16_t *peer_ports,
> +			       bool direction)
> +{
> +	struct rte_eth_dev *dev;
> +	int ret;
> +
> +	if (!peer_ports)
> +		return -EINVAL;
> +
> +	RTE_ETH_VALID_PORTID_OR_ERR_RET(cur_port, -EINVAL);
> +	dev = &rte_eth_devices[cur_port];
> +	RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops-
> >hairpin_get_peer_ports,
> +				-ENOTSUP);
> +
> +	ret = (*dev->dev_ops->hairpin_get_peer_ports)(dev, peer_ports,
> +						      direction);
> +	if (ret < 0)
> +		RTE_ETHDEV_LOG(ERR, "Failed to get %d hairpin peer %s
> ports",
> +			       cur_port, direction ? "RX" : "TX");
> +
> +	return ret;
> +}
> +
>  void
>  rte_eth_tx_buffer_drop_callback(struct rte_mbuf **pkts, uint16_t unsent,
>  		void *userdata __rte_unused)
> diff --git a/lib/librte_ethdev/rte_ethdev.h b/lib/librte_ethdev/rte_ethdev.h
> index 94a981c..6680de2 100644
> --- a/lib/librte_ethdev/rte_ethdev.h
> +++ b/lib/librte_ethdev/rte_ethdev.h
> @@ -2134,6 +2134,33 @@ int rte_eth_tx_queue_setup(uint16_t port_id,
> uint16_t tx_queue_id,
>   * @warning
>   * @b EXPERIMENTAL: this API may change, or be removed, without prior
> notice
>   *
> + * Get all the hairpin peer RX / TX ports of the current port.
> + * The caller should ensure that the array is large enough to save the ports
> + * list.
> + *
> + * @param cur_port
> + *   The current port identifier of the Ethernet device.
> + * @param peer_ports
> + *   Pointer to the array to save the peer ports list *
> + * @param direction
> + *   Current port to peer port direction
> + *   true - current used as TX to get all peer RX ports.
> + *   false - current used as RX to get all peer TX ports.
> + *
> + * @return
> + *   - (0 or positive) actual peer ports number.
> + *   - (-EINVAL) if bad parameter.
> + *   - (-ENOTSUP) if hardware doesn't support.
> + *   - Others detailed errors from PMD drivers.
> + */
> +__rte_experimental
> +int rte_eth_hairpin_get_peer_ports(uint16_t cur_port, uint16_t *peer_ports,
> +				   bool direction);
> +

I think direction should be changed to int, or change the name to ingress, also
I think we should add the peer_ports array size, so in case the application
didn't allocate enough space it will simply end in error.

> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change, or be removed, without prior
> notice
> + *
>   * Bind all hairpin TX queues of one port to the RX queues of the peer port.
>   * It is only allowed to call this API after all hairpin queues are configured
>   * properly and the devices of TX and peer RX are in started state.
> diff --git a/lib/librte_ethdev/rte_ethdev_driver.h
> b/lib/librte_ethdev/rte_ethdev_driver.h
> index f8eb879..ecca9d6 100644
> --- a/lib/librte_ethdev/rte_ethdev_driver.h
> +++ b/lib/librte_ethdev/rte_ethdev_driver.h
> @@ -576,6 +576,34 @@ typedef int (*eth_tx_hairpin_queue_setup_t)
> 
>  /**
>   * @internal
> + * Get all hairpin TX/RX peer ports of the current device, if any.
> + *
> + * @param dev
> + *   ethdev handle of port.
> + * @param peer_ports
> + *   array to save the ports list.
> + * @param direction
> + *   value to decide the current to peer direction
> + *   true - used as TX to get all peer RX ports.
> + *   false - used as RX to get all peer TX ports.
> + *
> + * @return
> + *   Negative errno value on error, 0 or positive on success.
> + *
> + * @retval 0
> + *   Success, no peer ports.
> + * @retval >0
> + *   Actual number of the peer ports.
> + * @retval -ENOTSUP
> + *   Get peer ports API is not supported.
> + * @retval -EINVAL
> + *   One of the parameters is invalid.
> + */
> +typedef int (*hairpin_get_peer_ports_t)(struct rte_eth_dev *dev,
> +					uint16_t *peer_ports, bool direction);
> +
> +/**
> + * @internal
>   * Bind all hairpin TX queues of one port to the RX queues of the peer port.
>   *
>   * @param dev
> @@ -761,6 +789,8 @@ struct eth_dev_ops {
>  	/**< Set up device RX hairpin queue. */
>  	eth_tx_hairpin_queue_setup_t tx_hairpin_queue_setup;
>  	/**< Set up device TX hairpin queue. */
> +	hairpin_get_peer_ports_t hairpin_get_peer_ports;
> +	/**< Get hairpin peer ports list. */
>  	eth_hairpin_bind_t hairpin_bind;
>  	/**< Bind all hairpin TX queues of device to the peer port RX queues. */
>  	eth_hairpin_unbind_t hairpin_unbind;
> diff --git a/lib/librte_ethdev/rte_ethdev_version.map
> b/lib/librte_ethdev/rte_ethdev_version.map
> index 18efe4e..1019e28 100644
> --- a/lib/librte_ethdev/rte_ethdev_version.map
> +++ b/lib/librte_ethdev/rte_ethdev_version.map
> @@ -228,6 +228,7 @@ EXPERIMENTAL {
> 
>  	# added in 20.11
>  	rte_eth_hairpin_bind;
> +	rte_eth_hairpin_get_peer_ports;
>  	rte_eth_hairpin_unbind;
>  	rte_eth_link_speed_to_str;
>  	rte_eth_link_to_str;
> --
> 1.8.3.1