From mboxrd@z Thu Jan  1 00:00:00 1970
From: "Rao, Nikhil" <nikhil.rao@intel.com>
Subject: Re: [PATCH 1/2] eventdev: add event adapter for ethernet
	Rx queues
Date: Thu, 27 Jul 2017 16:28:29 +0530
Message-ID: <123ed8d6-4fd9-8bee-d86e-d270a092169e@intel.com>
References: <29140c16-909a-1b9a-7391-481f900bd13c@intel.com>
 <1499377952-5306-1-git-send-email-nikhil.rao@intel.com>
 <20170706141829.GA5260@jerin>
 <02aef899-da84-9281-e4a4-2871237ea20e@intel.com>
 <20170707150317.GA2007@jerin> <20170707155707.GA6245@jerin>
 <3d2d78cc-9572-bf95-6d25-9b350da62827@intel.com>
 <20170710104126.GA13609@jerin>
 <4197b5f1-9a15-5892-12d2-6bd142bc4d85@intel.com>
 <20170713184445.GA3659@jerin>
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Cc: gage.eads@intel.com, dev@dpdk.org, thomas@monjalon.net,
 bruce.richardson@intel.com, harry.van.haaren@intel.com,
 hemant.agrawal@nxp.com, nipun.gupta@nxp.com, narender.vangati@intel.com,
 Abhinandan Gujjar <abhinandan.gujjar@intel.com>, nikhil.rao@intel.com
To: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Return-path: <dev-bounces@dpdk.org>
Received: from mga09.intel.com (mga09.intel.com [134.134.136.24])
 by dpdk.org (Postfix) with ESMTP id E355E7D8F
 for <dev@dpdk.org>; Thu, 27 Jul 2017 12:58:35 +0200 (CEST)
In-Reply-To: <20170713184445.GA3659@jerin>
Content-Language: en-US
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <http://dpdk.org/ml/options/dev>,
 <mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://dpdk.org/ml/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <http://dpdk.org/ml/listinfo/dev>,
 <mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org
Sender: "dev" <dev-bounces@dpdk.org>


Hi Jerin and all,

There are a few inconsistencies/complexities that I ran into with the 
implementation of the SW Rx event adapter, I have first summarized this 
email thread bringing together details scattered across various 
exchanges then I want to check if there are changes possible that would 
simplify the implementation of the SW Rx event adapter.

</start summary>
The Rx event adapter needs to support the following scenarios:

1) Ethdev HW is not capable of injecting the packets and SW eventdev
driver(All existing ethdev PMD + drivers/event/sw PMD combination)
2) Ethdev HW is not capable of injecting the packets and not compatible
HW eventdev driver(All existing ethdev PMD + driver/event/octeontx PMD
combination)
3) Ethdev HW is capable of injecting the packet to compatible
HW eventdev driver.

cases 1) and 2) above are not different. In both cases we need a SW 
thread that will inject packets from the ethdev PMD to an event dev PMD.

The APIs proposed are:

int rte_event_eth_rx_adapter_create(uint8_t dev_id,
         uint8_t eth_port_id, uint8_t id /* adapter ID */);

An adapter created above has an ops struct, where the ops
struct provides implementations for the functions below.
The ops struct chosen for the adapter depends on the <dev_id, 
eth_port_id> combination.

struct rte_event_eth_rx_adap_info {
         uint32_t cap;

/* adapter has inbuilt port, no need to create producer port */
#define RTE_EVENT_ETHDEV_CAP_INBUILT_PORT  (1ULL << 0)
/* adapter does not need service function */
#define RTE_EVENT_ETHDEV_CAP_NO_SERVICE_FUNC (1ULL << 1)

}
int rte_event_eth_rx_adapter_get_info(uint8_t dev_id, uint8_t id,
             struct rte_event_eth_rx_adap_info *info);

struct rte_event_eth_rx_adapter_conf {
	/* Application specified service function name */
	char service_name[];
	uint8_t rx_event_port_id;
          /**< Event port identifier, the adapter enqueues mbuf
        * events to this
            * port, Ignored when RTE_EVENT_ETHDEV_CAP_INBUILT_PORT
            */
  }
int rte_event_eth_rx_adapter_configure(uint8_t dev_id, uint8_t id
             struct rte_event_eth_rx_adapter_conf *cfg);

  struct rte_event_eth_rx_adapter_queue_conf {
     ... event info ...
          uint16_t servicing_weight;
     /**< Relative polling frequency of ethernet receive queue, if this
      * is set to zero, the Rx queue is interrupt driven (unless rx queue
      * interrupts are not enabled for the ethernet device)
      */
     ...
  }
int rte_event_eth_rx_adapter_queue_add(uint8_t dev_id,
             uint8_t id, int32_t rx_queue_id,
     const struct rte_eth_rx_event_adapter_queue_conf *config);

int rte_event_eth_rx_adapter_queue_del(uint8_t dev_id,
             uint8_t id, int32_t rx_queue_id)

</end summary>

In the case of a SW thread we would like to use the servicing weight 
specified in the queue to do WRR across <ports, queues[]>, in keeping 
with the adaper per <eventdev, eth port> model, one way to do this is to 
use the same cfg.service_name in the 
rte_event_eth_rx_adapter_configure() call.

However this creates a few difficulties/inconsistencies:

1)Service has the notion of a socket id. Multiple event dev IDs can be 
included in the same service, each event dev has a socket ID -> this 
seems to be an inconsistency that shouldn’t be allowed by design.

2)Say, the Rx event adapter doesn’t drop packets (could be 
configurable), i.e,  if events cannot be enqueued into the event device, 
these remain in a buffer, when the buffer fills up packets aren’t 
dequeued from the eth device.

In the simplest case the Rx event adapter service has a single <event 
device, event port> across multiple eth ports, it dequeues from the 
wrr[] and buffers events, bulk enqueues BATCH_SIZE events into the 
<event device, event port>.

With adapters having different <event device, event port> code can be 
optimized so that adapters that have a common <event device, event port> 
can be made to refer to a common enqueue buffer { event dev, event port, 
buffer } structure but this adds more book keeping in the code.

3)Every adapter can be configured with max_nb_rx ( a max nb of packets 
that it can process in any invocation) – but the max_nb_rx seems like a 
service level parameter instead of it being a summation across adapters.

1 & 3 could be solved by restricting the adapters to the same (as in the 
first rte_event_eth_rx_adapter_configure() call) socket ID, and perhaps 
using the max value of max_nb_rx or using the same value of max_nb_rx 
across adapters. #2 is doable but has a bit of code complexity to handle 
the generic case.

Before we go there, I wanted to check if there is an alternative 
possible that would remove the difficulties above. Essentially allow 
multiple ports within an adapter but avoid the problem of the 
inconsistent <eventdev, port> combinations when using multiple ports 
with a single eventdev.

Instead of
==
rte_event_eth_rx_adapter_create()
rte_event_eth_rx_adapter_get_info();
rte_event_eth_rx_adapter_configure();
rte_event_eth_rx_adapter_queue_add();
==

How about ?
==

rte_event_eth_rx_adapter_get_info(uint8_t dev_id, uint8_t eth_port_id,
         struct rte_event_eth_rx_adap_info *info);

struct rte_event_eth_rx_adap_info {
         uint32_t cap;

/* adapter has inbuilt port, no need to create producer port */
#define RTE_EVENT_ETHDEV_CAP_INBUILT_PORT  (1ULL << 0)
/* adapter does not need service function */
#define RTE_EVENT_ETHDEV_CAP_NO_SERVICE_FUNC (1ULL << 1)

}

rte_event_eth_rx_adapter_conf cfg;
cfg.event_port = event_port;
cfg.service_name = “rx_adapter_service”;

// all ports in eth_port_id[] have cap = 
//!RTE_EVENT_ETHDEV_CAP_INBUILT_PORT
// && ! RTE_EVENT_ETHDEV_CAP_NO_SERVICE_FUNC
rte_event_eth_rx_adapter_create(dev_id, eth_port_id[], N, id, &cfg);
===
int rte_event_eth_rx_adapter_queue_add() would need a port id in the N>1 
port case, that can be ignored if the adapter doesn’t need it (N=1).

thanks for reading the long email, thoughts ?

Nikhil