* [dpdk-dev] [PATCH v1 0/3] add more features for AF_XDP pmd
@ 2019-05-15 8:38 Xiaolong Ye
2019-05-15 8:38 ` [dpdk-dev] [PATCH v1 1/3] net/af_xdp: enable zero copy by extbuf Xiaolong Ye
` (3 more replies)
0 siblings, 4 replies; 16+ messages in thread
From: Xiaolong Ye @ 2019-05-15 8:38 UTC (permalink / raw)
Cc: Qi Zhang, Karlsson Magnus, Topel Bjorn, dev, Xiaolong Ye
Hi,
This patch series mainly includes 3 new features for AF_XDP pmd. They
are separated independent features, the reason I take them in one
patchset is that they have code dependency.
1. zero copy
This patch enables `zero copy` between af_xdp umem and mbuf by using
external mbuf mechanism.
2. multi-queue
With mutli-queue support, one AF_XDP pmd instance can use multi netdev
queues.
3. busy-poll
With busy-poll, all processing occurs on a single core, performance is
better from a per-core perspective.
This patch has dependency on busy-poll support in kernel side and now it is in
RFC stage [1].
[1] https://www.spinics.net/lists/netdev/msg568337.html
Xiaolong Ye (3):
net/af_xdp: enable zero copy by extbuf
net/af_xdp: add multi-queue support
net/af_xdp: add busy poll support
doc/guides/nics/af_xdp.rst | 5 +-
drivers/net/af_xdp/rte_eth_af_xdp.c | 210 +++++++++++++++++++++-------
2 files changed, 160 insertions(+), 55 deletions(-)
--
2.17.1
^ permalink raw reply [flat|nested] 16+ messages in thread* [dpdk-dev] [PATCH v1 1/3] net/af_xdp: enable zero copy by extbuf 2019-05-15 8:38 [dpdk-dev] [PATCH v1 0/3] add more features for AF_XDP pmd Xiaolong Ye @ 2019-05-15 8:38 ` Xiaolong Ye 2019-05-15 8:38 ` [dpdk-dev] [PATCH v1 2/3] net/af_xdp: add multi-queue support Xiaolong Ye ` (2 subsequent siblings) 3 siblings, 0 replies; 16+ messages in thread From: Xiaolong Ye @ 2019-05-15 8:38 UTC (permalink / raw) To: Xiaolong Ye, Qi Zhang, John McNamara, Marko Kovacevic Cc: Karlsson Magnus, Topel Bjorn, dev Implement zero copy of af_xdp pmd through mbuf's external memory mechanism to achieve high performance. This patch also provides a new parameter "pmd_zero_copy" for user, so they can choose to enable zero copy of af_xdp pmd or not. To be clear, "zero copy" here is different from the "zero copy mode" of AF_XDP, it is about zero copy between af_xdp umem and mbuf used in dpdk application. Suggested-by: Varghese Vipin <vipin.varghese@intel.com> Suggested-by: Olivier Matz <olivier.matz@6wind.com> Signed-off-by: Xiaolong Ye <xiaolong.ye@intel.com> --- doc/guides/nics/af_xdp.rst | 1 + drivers/net/af_xdp/rte_eth_af_xdp.c | 96 +++++++++++++++++++++++------ 2 files changed, 77 insertions(+), 20 deletions(-) diff --git a/doc/guides/nics/af_xdp.rst b/doc/guides/nics/af_xdp.rst index 175038153..0bd4239fe 100644 --- a/doc/guides/nics/af_xdp.rst +++ b/doc/guides/nics/af_xdp.rst @@ -28,6 +28,7 @@ The following options can be provided to set up an af_xdp port in DPDK. * ``iface`` - name of the Kernel interface to attach to (required); * ``queue`` - netdev queue id (optional, default 0); +* ``pmd_zero_copy`` - enable zero copy or not (optional, default 0); Prerequisites ------------- diff --git a/drivers/net/af_xdp/rte_eth_af_xdp.c b/drivers/net/af_xdp/rte_eth_af_xdp.c index 35c72272c..ebef7bf34 100644 --- a/drivers/net/af_xdp/rte_eth_af_xdp.c +++ b/drivers/net/af_xdp/rte_eth_af_xdp.c @@ -70,6 +70,7 @@ struct xsk_umem_info { struct xsk_umem *umem; struct rte_ring *buf_ring; const struct rte_memzone *mz; + int pmd_zc; }; struct rx_stats { @@ -109,6 +110,7 @@ struct pmd_internals { int if_index; char if_name[IFNAMSIZ]; uint16_t queue_idx; + int pmd_zc; struct ether_addr eth_addr; struct xsk_umem_info *umem; struct rte_mempool *mb_pool_share; @@ -119,10 +121,12 @@ struct pmd_internals { #define ETH_AF_XDP_IFACE_ARG "iface" #define ETH_AF_XDP_QUEUE_IDX_ARG "queue" +#define ETH_AF_XDP_PMD_ZC_ARG "pmd_zero_copy" static const char * const valid_arguments[] = { ETH_AF_XDP_IFACE_ARG, ETH_AF_XDP_QUEUE_IDX_ARG, + ETH_AF_XDP_PMD_ZC_ARG, NULL }; @@ -166,6 +170,15 @@ reserve_fill_queue(struct xsk_umem_info *umem, uint16_t reserve_size) return 0; } +static void +umem_buf_release_to_fq(void *addr, void *opaque) +{ + struct xsk_umem_info *umem = (struct xsk_umem_info *)opaque; + uint64_t umem_addr = (uint64_t)addr - umem->mz->addr_64; + + rte_ring_enqueue(umem->buf_ring, (void *)umem_addr); +} + static uint16_t eth_af_xdp_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) { @@ -175,6 +188,7 @@ eth_af_xdp_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) struct xsk_ring_prod *fq = &umem->fq; uint32_t idx_rx = 0; uint32_t free_thresh = fq->size >> 1; + int pmd_zc = umem->pmd_zc; struct rte_mbuf *mbufs[ETH_AF_XDP_RX_BATCH_SIZE]; unsigned long dropped = 0; unsigned long rx_bytes = 0; @@ -197,19 +211,29 @@ eth_af_xdp_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) uint64_t addr; uint32_t len; void *pkt; + uint16_t buf_len = ETH_AF_XDP_FRAME_SIZE; + struct rte_mbuf_ext_shared_info *shinfo; desc = xsk_ring_cons__rx_desc(rx, idx_rx++); addr = desc->addr; len = desc->len; pkt = xsk_umem__get_data(rxq->umem->mz->addr, addr); - rte_memcpy(rte_pktmbuf_mtod(mbufs[i], void *), pkt, len); + if (pmd_zc) { + shinfo = rte_pktmbuf_ext_shinfo_init_helper(pkt, + &buf_len, umem_buf_release_to_fq, umem); + + rte_pktmbuf_attach_extbuf(mbufs[i], pkt, 0, buf_len, + shinfo); + } else { + rte_memcpy(rte_pktmbuf_mtod(mbufs[i], void *), + pkt, len); + rte_ring_enqueue(umem->buf_ring, (void *)addr); + } rte_pktmbuf_pkt_len(mbufs[i]) = len; rte_pktmbuf_data_len(mbufs[i]) = len; rx_bytes += len; bufs[i] = mbufs[i]; - - rte_ring_enqueue(umem->buf_ring, (void *)addr); } xsk_ring_cons__release(rx, rcvd); @@ -262,12 +286,21 @@ kick_tx(struct pkt_tx_queue *txq) pull_umem_cq(umem, ETH_AF_XDP_TX_BATCH_SIZE); } +static inline bool +in_umem_range(struct xsk_umem_info *umem, uint64_t addr) +{ + uint64_t mz_base_addr = umem->mz->addr_64; + + return addr >= mz_base_addr && addr < mz_base_addr + umem->mz->len; +} + static uint16_t eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) { struct pkt_tx_queue *txq = queue; struct xsk_umem_info *umem = txq->pair->umem; struct rte_mbuf *mbuf; + int pmd_zc = umem->pmd_zc; void *addrs[ETH_AF_XDP_TX_BATCH_SIZE]; unsigned long tx_bytes = 0; int i; @@ -294,16 +327,26 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) desc = xsk_ring_prod__tx_desc(&txq->tx, idx_tx + i); mbuf = bufs[i]; - - desc->addr = (uint64_t)addrs[i]; desc->len = mbuf->pkt_len; - pkt = xsk_umem__get_data(umem->mz->addr, - desc->addr); - rte_memcpy(pkt, rte_pktmbuf_mtod(mbuf, void *), - desc->len); - tx_bytes += mbuf->pkt_len; - rte_pktmbuf_free(mbuf); + /* + * We need to make sure the external mbuf address is within + * current port's umem memzone range + */ + if (pmd_zc && RTE_MBUF_HAS_EXTBUF(mbuf) && + in_umem_range(umem, (uint64_t)mbuf->buf_addr)) { + desc->addr = (uint64_t)mbuf->buf_addr - + umem->mz->addr_64; + mbuf->buf_addr = xsk_umem__get_data(umem->mz->addr, + (uint64_t)addrs[i]); + } else { + desc->addr = (uint64_t)addrs[i]; + pkt = xsk_umem__get_data(umem->mz->addr, + desc->addr); + rte_memcpy(pkt, rte_pktmbuf_mtod(mbuf, void *), + desc->len); + } + tx_bytes += mbuf->pkt_len; } xsk_ring_prod__submit(&txq->tx, nb_pkts); @@ -313,6 +356,9 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) txq->stats.tx_pkts += nb_pkts; txq->stats.tx_bytes += tx_bytes; + for (i = 0; i < nb_pkts; i++) + rte_pktmbuf_free(bufs[i]); + return nb_pkts; } @@ -640,6 +686,10 @@ eth_rx_queue_setup(struct rte_eth_dev *dev, } internals->umem = rxq->umem; + internals->umem->pmd_zc = internals->pmd_zc; + + if (internals->umem->pmd_zc) + AF_XDP_LOG(INFO, "Zero copy between umem and mbuf enabled.\n"); dev->data->rx_queues[rx_queue_id] = rxq; return 0; @@ -775,9 +825,8 @@ parse_name_arg(const char *key __rte_unused, } static int -parse_parameters(struct rte_kvargs *kvlist, - char *if_name, - int *queue_idx) +parse_parameters(struct rte_kvargs *kvlist, char *if_name, int *queue_idx, + int *pmd_zc) { int ret; @@ -791,6 +840,11 @@ parse_parameters(struct rte_kvargs *kvlist, if (ret < 0) goto free_kvlist; + ret = rte_kvargs_process(kvlist, ETH_AF_XDP_PMD_ZC_ARG, + &parse_integer_arg, pmd_zc); + if (ret < 0) + goto free_kvlist; + free_kvlist: rte_kvargs_free(kvlist); return ret; @@ -827,9 +881,8 @@ get_iface_info(const char *if_name, } static struct rte_eth_dev * -init_internals(struct rte_vdev_device *dev, - const char *if_name, - int queue_idx) +init_internals(struct rte_vdev_device *dev, const char *if_name, int queue_idx, + int pmd_zc) { const char *name = rte_vdev_device_name(dev); const unsigned int numa_node = dev->device.numa_node; @@ -843,6 +896,7 @@ init_internals(struct rte_vdev_device *dev, return NULL; internals->queue_idx = queue_idx; + internals->pmd_zc = pmd_zc; strlcpy(internals->if_name, if_name, IFNAMSIZ); for (i = 0; i < ETH_AF_XDP_MAX_QUEUE_PAIRS; i++) { @@ -883,6 +937,7 @@ rte_pmd_af_xdp_probe(struct rte_vdev_device *dev) int xsk_queue_idx = ETH_AF_XDP_DFLT_QUEUE_IDX; struct rte_eth_dev *eth_dev = NULL; const char *name; + int pmd_zc = 0; AF_XDP_LOG(INFO, "Initializing pmd_af_xdp for %s\n", rte_vdev_device_name(dev)); @@ -909,7 +964,7 @@ rte_pmd_af_xdp_probe(struct rte_vdev_device *dev) if (dev->device.numa_node == SOCKET_ID_ANY) dev->device.numa_node = rte_socket_id(); - if (parse_parameters(kvlist, if_name, &xsk_queue_idx) < 0) { + if (parse_parameters(kvlist, if_name, &xsk_queue_idx, &pmd_zc) < 0) { AF_XDP_LOG(ERR, "Invalid kvargs value\n"); return -EINVAL; } @@ -919,7 +974,7 @@ rte_pmd_af_xdp_probe(struct rte_vdev_device *dev) return -EINVAL; } - eth_dev = init_internals(dev, if_name, xsk_queue_idx); + eth_dev = init_internals(dev, if_name, xsk_queue_idx, pmd_zc); if (eth_dev == NULL) { AF_XDP_LOG(ERR, "Failed to init internals\n"); return -1; @@ -961,7 +1016,8 @@ static struct rte_vdev_driver pmd_af_xdp_drv = { RTE_PMD_REGISTER_VDEV(net_af_xdp, pmd_af_xdp_drv); RTE_PMD_REGISTER_PARAM_STRING(net_af_xdp, "iface=<string> " - "queue=<int> "); + "queue=<int> " + "pmd_zero_copy=<0|1>"); RTE_INIT(af_xdp_init_log) { -- 2.17.1 ^ permalink raw reply related [flat|nested] 16+ messages in thread
* [dpdk-dev] [PATCH v1 2/3] net/af_xdp: add multi-queue support 2019-05-15 8:38 [dpdk-dev] [PATCH v1 0/3] add more features for AF_XDP pmd Xiaolong Ye 2019-05-15 8:38 ` [dpdk-dev] [PATCH v1 1/3] net/af_xdp: enable zero copy by extbuf Xiaolong Ye @ 2019-05-15 8:38 ` Xiaolong Ye 2019-05-15 8:38 ` [dpdk-dev] [PATCH v1 3/3] net/af_xdp: add busy poll support Xiaolong Ye 2019-05-30 9:07 ` [dpdk-dev] [PATCH v2 0/3] add more features for AF_XDP pmd Xiaolong Ye 3 siblings, 0 replies; 16+ messages in thread From: Xiaolong Ye @ 2019-05-15 8:38 UTC (permalink / raw) To: Xiaolong Ye, Qi Zhang, John McNamara, Marko Kovacevic Cc: Karlsson Magnus, Topel Bjorn, dev This patch adds two parameters `start_queue` and `queue_count` to specify the range of netdev queues used by AF_XDP pmd. Signed-off-by: Xiaolong Ye <xiaolong.ye@intel.com> --- doc/guides/nics/af_xdp.rst | 3 +- drivers/net/af_xdp/rte_eth_af_xdp.c | 88 +++++++++++++++-------------- 2 files changed, 49 insertions(+), 42 deletions(-) diff --git a/doc/guides/nics/af_xdp.rst b/doc/guides/nics/af_xdp.rst index 0bd4239fe..18defcda3 100644 --- a/doc/guides/nics/af_xdp.rst +++ b/doc/guides/nics/af_xdp.rst @@ -27,7 +27,8 @@ Options The following options can be provided to set up an af_xdp port in DPDK. * ``iface`` - name of the Kernel interface to attach to (required); -* ``queue`` - netdev queue id (optional, default 0); +* ``start_queue`` - starting netdev queue id (optional, default 0); +* ``queue_count`` - total netdev queue number (optional, default 1); * ``pmd_zero_copy`` - enable zero copy or not (optional, default 0); Prerequisites diff --git a/drivers/net/af_xdp/rte_eth_af_xdp.c b/drivers/net/af_xdp/rte_eth_af_xdp.c index ebef7bf34..9a4510701 100644 --- a/drivers/net/af_xdp/rte_eth_af_xdp.c +++ b/drivers/net/af_xdp/rte_eth_af_xdp.c @@ -57,7 +57,8 @@ static int af_xdp_logtype; #define ETH_AF_XDP_NUM_BUFFERS 4096 #define ETH_AF_XDP_DATA_HEADROOM 0 #define ETH_AF_XDP_DFLT_NUM_DESCS XSK_RING_CONS__DEFAULT_NUM_DESCS -#define ETH_AF_XDP_DFLT_QUEUE_IDX 0 +#define ETH_AF_XDP_DFLT_START_QUEUE_IDX 0 +#define ETH_AF_XDP_DFLT_QUEUE_COUNT 1 #define ETH_AF_XDP_RX_BATCH_SIZE 32 #define ETH_AF_XDP_TX_BATCH_SIZE 32 @@ -88,7 +89,7 @@ struct pkt_rx_queue { struct rx_stats stats; struct pkt_tx_queue *pair; - uint16_t queue_idx; + int xsk_queue_idx; }; struct tx_stats { @@ -103,13 +104,15 @@ struct pkt_tx_queue { struct tx_stats stats; struct pkt_rx_queue *pair; - uint16_t queue_idx; + int xsk_queue_idx; }; struct pmd_internals { int if_index; char if_name[IFNAMSIZ]; - uint16_t queue_idx; + int start_queue_idx; + int queue_cnt; + int pmd_zc; struct ether_addr eth_addr; struct xsk_umem_info *umem; @@ -120,12 +123,14 @@ struct pmd_internals { }; #define ETH_AF_XDP_IFACE_ARG "iface" -#define ETH_AF_XDP_QUEUE_IDX_ARG "queue" +#define ETH_AF_XDP_START_QUEUE_ARG "start_queue" +#define ETH_AF_XDP_QUEUE_COUNT_ARG "queue_count" #define ETH_AF_XDP_PMD_ZC_ARG "pmd_zero_copy" static const char * const valid_arguments[] = { ETH_AF_XDP_IFACE_ARG, - ETH_AF_XDP_QUEUE_IDX_ARG, + ETH_AF_XDP_START_QUEUE_ARG, + ETH_AF_XDP_QUEUE_COUNT_ARG, ETH_AF_XDP_PMD_ZC_ARG, NULL }; @@ -395,8 +400,8 @@ eth_dev_info(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info) dev_info->if_index = internals->if_index; dev_info->max_mac_addrs = 1; dev_info->max_rx_pktlen = ETH_FRAME_LEN; - dev_info->max_rx_queues = 1; - dev_info->max_tx_queues = 1; + dev_info->max_rx_queues = internals->queue_cnt; + dev_info->max_tx_queues = internals->queue_cnt; dev_info->min_mtu = ETHER_MIN_MTU; dev_info->max_mtu = ETH_AF_XDP_FRAME_SIZE - ETH_AF_XDP_DATA_HEADROOM; @@ -528,7 +533,8 @@ eth_link_update(struct rte_eth_dev *dev __rte_unused, } static struct -xsk_umem_info *xdp_umem_configure(struct pmd_internals *internals) +xsk_umem_info *xdp_umem_configure(struct pmd_internals *internals, + struct pkt_rx_queue *rxq) { struct xsk_umem_info *umem; const struct rte_memzone *mz; @@ -549,7 +555,7 @@ xsk_umem_info *xdp_umem_configure(struct pmd_internals *internals) } snprintf(ring_name, sizeof(ring_name), "af_xdp_ring_%s_%u", - internals->if_name, internals->queue_idx); + internals->if_name, rxq->xsk_queue_idx); umem->buf_ring = rte_ring_create(ring_name, ETH_AF_XDP_NUM_BUFFERS, rte_socket_id(), @@ -565,7 +571,7 @@ xsk_umem_info *xdp_umem_configure(struct pmd_internals *internals) ETH_AF_XDP_DATA_HEADROOM)); snprintf(mz_name, sizeof(mz_name), "af_xdp_umem_%s_%u", - internals->if_name, internals->queue_idx); + internals->if_name, rxq->xsk_queue_idx); mz = rte_memzone_reserve_aligned(mz_name, ETH_AF_XDP_NUM_BUFFERS * ETH_AF_XDP_FRAME_SIZE, rte_socket_id(), RTE_MEMZONE_IOVA_CONTIG, @@ -602,7 +608,7 @@ xsk_configure(struct pmd_internals *internals, struct pkt_rx_queue *rxq, int ret = 0; int reserve_size; - rxq->umem = xdp_umem_configure(internals); + rxq->umem = xdp_umem_configure(internals, rxq); if (rxq->umem == NULL) return -ENOMEM; @@ -612,7 +618,7 @@ xsk_configure(struct pmd_internals *internals, struct pkt_rx_queue *rxq, cfg.xdp_flags = XDP_FLAGS_UPDATE_IF_NOEXIST; cfg.bind_flags = 0; ret = xsk_socket__create(&rxq->xsk, internals->if_name, - internals->queue_idx, rxq->umem->umem, &rxq->rx, + rxq->xsk_queue_idx, rxq->umem->umem, &rxq->rx, &txq->tx, &cfg); if (ret) { AF_XDP_LOG(ERR, "Failed to create xsk socket.\n"); @@ -635,20 +641,6 @@ xsk_configure(struct pmd_internals *internals, struct pkt_rx_queue *rxq, return ret; } -static void -queue_reset(struct pmd_internals *internals, uint16_t queue_idx) -{ - struct pkt_rx_queue *rxq = &internals->rx_queues[queue_idx]; - struct pkt_tx_queue *txq = rxq->pair; - - memset(rxq, 0, sizeof(*rxq)); - memset(txq, 0, sizeof(*txq)); - rxq->pair = txq; - txq->pair = rxq; - rxq->queue_idx = queue_idx; - txq->queue_idx = queue_idx; -} - static int eth_rx_queue_setup(struct rte_eth_dev *dev, uint16_t rx_queue_id, @@ -663,8 +655,9 @@ eth_rx_queue_setup(struct rte_eth_dev *dev, int ret; rxq = &internals->rx_queues[rx_queue_id]; - queue_reset(internals, rx_queue_id); + AF_XDP_LOG(INFO, "Set up rx queue, rx queue id: %d, xsk queue id: %d\n", + rx_queue_id, rxq->xsk_queue_idx); /* Now get the space available for data in the mbuf */ buf_size = rte_pktmbuf_data_room_size(mb_pool) - RTE_PKTMBUF_HEADROOM; @@ -695,7 +688,6 @@ eth_rx_queue_setup(struct rte_eth_dev *dev, return 0; err: - queue_reset(internals, rx_queue_id); return ret; } @@ -825,8 +817,8 @@ parse_name_arg(const char *key __rte_unused, } static int -parse_parameters(struct rte_kvargs *kvlist, char *if_name, int *queue_idx, - int *pmd_zc) +parse_parameters(struct rte_kvargs *kvlist, char *if_name, int *start_queue, + int *queue_cnt, int *pmd_zc) { int ret; @@ -835,11 +827,18 @@ parse_parameters(struct rte_kvargs *kvlist, char *if_name, int *queue_idx, if (ret < 0) goto free_kvlist; - ret = rte_kvargs_process(kvlist, ETH_AF_XDP_QUEUE_IDX_ARG, - &parse_integer_arg, queue_idx); + ret = rte_kvargs_process(kvlist, ETH_AF_XDP_START_QUEUE_ARG, + &parse_integer_arg, start_queue); if (ret < 0) goto free_kvlist; + ret = rte_kvargs_process(kvlist, ETH_AF_XDP_QUEUE_COUNT_ARG, + &parse_integer_arg, queue_cnt); + if (ret < 0 || *queue_cnt > ETH_AF_XDP_MAX_QUEUE_PAIRS) { + ret = -EINVAL; + goto free_kvlist; + } + ret = rte_kvargs_process(kvlist, ETH_AF_XDP_PMD_ZC_ARG, &parse_integer_arg, pmd_zc); if (ret < 0) @@ -881,8 +880,8 @@ get_iface_info(const char *if_name, } static struct rte_eth_dev * -init_internals(struct rte_vdev_device *dev, const char *if_name, int queue_idx, - int pmd_zc) +init_internals(struct rte_vdev_device *dev, const char *if_name, + int start_queue_idx, int queue_cnt, int pmd_zc) { const char *name = rte_vdev_device_name(dev); const unsigned int numa_node = dev->device.numa_node; @@ -895,13 +894,16 @@ init_internals(struct rte_vdev_device *dev, const char *if_name, int queue_idx, if (internals == NULL) return NULL; - internals->queue_idx = queue_idx; + internals->start_queue_idx = start_queue_idx; + internals->queue_cnt = queue_cnt; internals->pmd_zc = pmd_zc; strlcpy(internals->if_name, if_name, IFNAMSIZ); - for (i = 0; i < ETH_AF_XDP_MAX_QUEUE_PAIRS; i++) { + for (i = 0; i < queue_cnt; i++) { internals->tx_queues[i].pair = &internals->rx_queues[i]; internals->rx_queues[i].pair = &internals->tx_queues[i]; + internals->rx_queues[i].xsk_queue_idx = start_queue_idx + i; + internals->tx_queues[i].xsk_queue_idx = start_queue_idx + i; } ret = get_iface_info(if_name, &internals->eth_addr, @@ -934,7 +936,8 @@ rte_pmd_af_xdp_probe(struct rte_vdev_device *dev) { struct rte_kvargs *kvlist; char if_name[IFNAMSIZ] = {'\0'}; - int xsk_queue_idx = ETH_AF_XDP_DFLT_QUEUE_IDX; + int xsk_start_queue_idx = ETH_AF_XDP_DFLT_START_QUEUE_IDX; + int xsk_queue_cnt = ETH_AF_XDP_DFLT_QUEUE_COUNT; struct rte_eth_dev *eth_dev = NULL; const char *name; int pmd_zc = 0; @@ -964,7 +967,8 @@ rte_pmd_af_xdp_probe(struct rte_vdev_device *dev) if (dev->device.numa_node == SOCKET_ID_ANY) dev->device.numa_node = rte_socket_id(); - if (parse_parameters(kvlist, if_name, &xsk_queue_idx, &pmd_zc) < 0) { + if (parse_parameters(kvlist, if_name, &xsk_start_queue_idx, + &xsk_queue_cnt, &pmd_zc) < 0) { AF_XDP_LOG(ERR, "Invalid kvargs value\n"); return -EINVAL; } @@ -974,7 +978,8 @@ rte_pmd_af_xdp_probe(struct rte_vdev_device *dev) return -EINVAL; } - eth_dev = init_internals(dev, if_name, xsk_queue_idx, pmd_zc); + eth_dev = init_internals(dev, if_name, xsk_start_queue_idx, + xsk_queue_cnt, pmd_zc); if (eth_dev == NULL) { AF_XDP_LOG(ERR, "Failed to init internals\n"); return -1; @@ -1016,7 +1021,8 @@ static struct rte_vdev_driver pmd_af_xdp_drv = { RTE_PMD_REGISTER_VDEV(net_af_xdp, pmd_af_xdp_drv); RTE_PMD_REGISTER_PARAM_STRING(net_af_xdp, "iface=<string> " - "queue=<int> " + "start_queue=<int> " + "queue_count=<int> " "pmd_zero_copy=<0|1>"); RTE_INIT(af_xdp_init_log) -- 2.17.1 ^ permalink raw reply related [flat|nested] 16+ messages in thread
* [dpdk-dev] [PATCH v1 3/3] net/af_xdp: add busy poll support 2019-05-15 8:38 [dpdk-dev] [PATCH v1 0/3] add more features for AF_XDP pmd Xiaolong Ye 2019-05-15 8:38 ` [dpdk-dev] [PATCH v1 1/3] net/af_xdp: enable zero copy by extbuf Xiaolong Ye 2019-05-15 8:38 ` [dpdk-dev] [PATCH v1 2/3] net/af_xdp: add multi-queue support Xiaolong Ye @ 2019-05-15 8:38 ` Xiaolong Ye 2019-05-30 9:07 ` [dpdk-dev] [PATCH v2 0/3] add more features for AF_XDP pmd Xiaolong Ye 3 siblings, 0 replies; 16+ messages in thread From: Xiaolong Ye @ 2019-05-15 8:38 UTC (permalink / raw) To: Xiaolong Ye, Qi Zhang, John McNamara, Marko Kovacevic Cc: Karlsson Magnus, Topel Bjorn, dev This patch enables busy-poll support for AF_XDP pmd. With busy-poll, the kernel driver is executed in process context by calling the poll() syscall. The main advantage of busy-poll feature is that all processing occurs on a single core. This eliminates the core-to-core cache transfers that occur between the application and the softirqd processing on another core. The drawback of busy-poll is that it will downgrade the max throughput due to syscall, but from a per-core perspective, the performance is better as normal mode runs on two cores and busy-poll only runs on a single core. Signed-off-by: Xiaolong Ye <xiaolong.ye@intel.com> --- doc/guides/nics/af_xdp.rst | 1 + drivers/net/af_xdp/rte_eth_af_xdp.c | 48 ++++++++++++++++++++++++++--- 2 files changed, 45 insertions(+), 4 deletions(-) diff --git a/doc/guides/nics/af_xdp.rst b/doc/guides/nics/af_xdp.rst index 18defcda3..e42065170 100644 --- a/doc/guides/nics/af_xdp.rst +++ b/doc/guides/nics/af_xdp.rst @@ -29,6 +29,7 @@ The following options can be provided to set up an af_xdp port in DPDK. * ``iface`` - name of the Kernel interface to attach to (required); * ``start_queue`` - starting netdev queue id (optional, default 0); * ``queue_count`` - total netdev queue number (optional, default 1); +* ``busy_poll_size`` - busy poll batch size (optional, default 0); * ``pmd_zero_copy`` - enable zero copy or not (optional, default 0); Prerequisites diff --git a/drivers/net/af_xdp/rte_eth_af_xdp.c b/drivers/net/af_xdp/rte_eth_af_xdp.c index 9a4510701..1e46a4ef4 100644 --- a/drivers/net/af_xdp/rte_eth_af_xdp.c +++ b/drivers/net/af_xdp/rte_eth_af_xdp.c @@ -6,6 +6,7 @@ #include <stdlib.h> #include <string.h> #include <netinet/in.h> +#include <poll.h> #include <net/if.h> #include <sys/socket.h> #include <sys/ioctl.h> @@ -72,6 +73,7 @@ struct xsk_umem_info { struct rte_ring *buf_ring; const struct rte_memzone *mz; int pmd_zc; + int busy_poll; }; struct rx_stats { @@ -114,6 +116,7 @@ struct pmd_internals { int queue_cnt; int pmd_zc; + int busy_poll_size; struct ether_addr eth_addr; struct xsk_umem_info *umem; struct rte_mempool *mb_pool_share; @@ -126,12 +129,14 @@ struct pmd_internals { #define ETH_AF_XDP_START_QUEUE_ARG "start_queue" #define ETH_AF_XDP_QUEUE_COUNT_ARG "queue_count" #define ETH_AF_XDP_PMD_ZC_ARG "pmd_zero_copy" +#define ETH_AF_XDP_BUSY_POLL_SIZE_ARG "busy_poll_size" static const char * const valid_arguments[] = { ETH_AF_XDP_IFACE_ARG, ETH_AF_XDP_START_QUEUE_ARG, ETH_AF_XDP_QUEUE_COUNT_ARG, ETH_AF_XDP_PMD_ZC_ARG, + ETH_AF_XDP_BUSY_POLL_SIZE_ARG, NULL }; @@ -191,6 +196,7 @@ eth_af_xdp_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) struct xsk_ring_cons *rx = &rxq->rx; struct xsk_umem_info *umem = rxq->umem; struct xsk_ring_prod *fq = &umem->fq; + struct pollfd pfds[1]; uint32_t idx_rx = 0; uint32_t free_thresh = fq->size >> 1; int pmd_zc = umem->pmd_zc; @@ -199,6 +205,15 @@ eth_af_xdp_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) unsigned long rx_bytes = 0; int rcvd, i; + if (umem->busy_poll) { + memset(pfds, 0, sizeof(pfds)); + pfds[0].fd = xsk_socket__fd(rxq->xsk); + pfds[0].events = POLLIN; + + if (poll(pfds, 1, 0) <= 0) + return 0; + } + nb_pkts = RTE_MIN(nb_pkts, ETH_AF_XDP_RX_BATCH_SIZE); if (unlikely(rte_pktmbuf_alloc_bulk(rxq->mb_pool, mbufs, nb_pkts) != 0)) @@ -305,12 +320,23 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) struct pkt_tx_queue *txq = queue; struct xsk_umem_info *umem = txq->pair->umem; struct rte_mbuf *mbuf; + struct pollfd pfds[1]; int pmd_zc = umem->pmd_zc; void *addrs[ETH_AF_XDP_TX_BATCH_SIZE]; unsigned long tx_bytes = 0; int i; uint32_t idx_tx; + if (umem->busy_poll) { + memset(pfds, 0, sizeof(pfds)); + pfds[0].fd = xsk_socket__fd(txq->pair->xsk); + pfds[0].events = POLLOUT; + if (poll(pfds, 1, 0) <= 0) + return 0; + if (!(pfds[0].revents & POLLOUT)) + return 0; + } + nb_pkts = RTE_MIN(nb_pkts, ETH_AF_XDP_TX_BATCH_SIZE); pull_umem_cq(umem, nb_pkts); @@ -615,6 +641,7 @@ xsk_configure(struct pmd_internals *internals, struct pkt_rx_queue *rxq, cfg.rx_size = ring_size; cfg.tx_size = ring_size; cfg.libbpf_flags = 0; + cfg.busy_poll = internals->busy_poll_size; cfg.xdp_flags = XDP_FLAGS_UPDATE_IF_NOEXIST; cfg.bind_flags = 0; ret = xsk_socket__create(&rxq->xsk, internals->if_name, @@ -680,10 +707,14 @@ eth_rx_queue_setup(struct rte_eth_dev *dev, internals->umem = rxq->umem; internals->umem->pmd_zc = internals->pmd_zc; + internals->umem->busy_poll = internals->busy_poll_size ? 1 : 0; if (internals->umem->pmd_zc) AF_XDP_LOG(INFO, "Zero copy between umem and mbuf enabled.\n"); + if (internals->umem->busy_poll) + AF_XDP_LOG(INFO, "Busy poll enabled.\n"); + dev->data->rx_queues[rx_queue_id] = rxq; return 0; @@ -818,7 +849,7 @@ parse_name_arg(const char *key __rte_unused, static int parse_parameters(struct rte_kvargs *kvlist, char *if_name, int *start_queue, - int *queue_cnt, int *pmd_zc) + int *queue_cnt, int *pmd_zc, int *busy_poll_size) { int ret; @@ -844,6 +875,11 @@ parse_parameters(struct rte_kvargs *kvlist, char *if_name, int *start_queue, if (ret < 0) goto free_kvlist; + ret = rte_kvargs_process(kvlist, ETH_AF_XDP_BUSY_POLL_SIZE_ARG, + &parse_integer_arg, busy_poll_size); + if (ret < 0) + goto free_kvlist; + free_kvlist: rte_kvargs_free(kvlist); return ret; @@ -881,7 +917,8 @@ get_iface_info(const char *if_name, static struct rte_eth_dev * init_internals(struct rte_vdev_device *dev, const char *if_name, - int start_queue_idx, int queue_cnt, int pmd_zc) + int start_queue_idx, int queue_cnt, int pmd_zc, + int busy_poll_size) { const char *name = rte_vdev_device_name(dev); const unsigned int numa_node = dev->device.numa_node; @@ -897,6 +934,7 @@ init_internals(struct rte_vdev_device *dev, const char *if_name, internals->start_queue_idx = start_queue_idx; internals->queue_cnt = queue_cnt; internals->pmd_zc = pmd_zc; + internals->busy_poll_size = busy_poll_size; strlcpy(internals->if_name, if_name, IFNAMSIZ); for (i = 0; i < queue_cnt; i++) { @@ -941,6 +979,7 @@ rte_pmd_af_xdp_probe(struct rte_vdev_device *dev) struct rte_eth_dev *eth_dev = NULL; const char *name; int pmd_zc = 0; + int busy_poll_size = 0; AF_XDP_LOG(INFO, "Initializing pmd_af_xdp for %s\n", rte_vdev_device_name(dev)); @@ -968,7 +1007,7 @@ rte_pmd_af_xdp_probe(struct rte_vdev_device *dev) dev->device.numa_node = rte_socket_id(); if (parse_parameters(kvlist, if_name, &xsk_start_queue_idx, - &xsk_queue_cnt, &pmd_zc) < 0) { + &xsk_queue_cnt, &pmd_zc, &busy_poll_size) < 0) { AF_XDP_LOG(ERR, "Invalid kvargs value\n"); return -EINVAL; } @@ -979,7 +1018,7 @@ rte_pmd_af_xdp_probe(struct rte_vdev_device *dev) } eth_dev = init_internals(dev, if_name, xsk_start_queue_idx, - xsk_queue_cnt, pmd_zc); + xsk_queue_cnt, pmd_zc, busy_poll_size); if (eth_dev == NULL) { AF_XDP_LOG(ERR, "Failed to init internals\n"); return -1; @@ -1023,6 +1062,7 @@ RTE_PMD_REGISTER_PARAM_STRING(net_af_xdp, "iface=<string> " "start_queue=<int> " "queue_count=<int> " + "busy_poll_size=<int> " "pmd_zero_copy=<0|1>"); RTE_INIT(af_xdp_init_log) -- 2.17.1 ^ permalink raw reply related [flat|nested] 16+ messages in thread
* [dpdk-dev] [PATCH v2 0/3] add more features for AF_XDP pmd 2019-05-15 8:38 [dpdk-dev] [PATCH v1 0/3] add more features for AF_XDP pmd Xiaolong Ye ` (2 preceding siblings ...) 2019-05-15 8:38 ` [dpdk-dev] [PATCH v1 3/3] net/af_xdp: add busy poll support Xiaolong Ye @ 2019-05-30 9:07 ` Xiaolong Ye 2019-05-30 9:07 ` [dpdk-dev] [PATCH v2 1/3] net/af_xdp: enable zero copy by extbuf Xiaolong Ye ` (3 more replies) 3 siblings, 4 replies; 16+ messages in thread From: Xiaolong Ye @ 2019-05-30 9:07 UTC (permalink / raw) Cc: Qi Zhang, Karlsson Magnus, Topel Bjorn, dev, Xiaolong Ye This patch series mainly includes 3 new features for AF_XDP pmd. They are separated independent features, the reason I take them in one patchset is that they have code dependency. 1. zero copy This patch enables `zero copy` between af_xdp umem and mbuf by using external mbuf mechanism. 2. multi-queue With mutli-queue support, one AF_XDP pmd instance can use multi netdev queues. 3. busy-poll With busy-poll, all processing occurs on a single core, performance is better from a per-core perspective. This patch has dependency on busy-poll support in kernel side and now it is in RFC stage [1]. [1] https://www.spinics.net/lists/netdev/msg568337.html V2 changes: 1. improve mutli-queue support by getting the ethtool channel, so the driver is able to get a reason maximum queue number. 2. add a tiny cleanup patch to get rid of unused struct member 3. remove the busy-poll patch as its kernel counterpart changes, will update the patch later Xiaolong Ye (3): net/af_xdp: enable zero copy by extbuf net/af_xdp: add multi-queue support net/af_xdp: remove unused struct member doc/guides/nics/af_xdp.rst | 4 +- drivers/net/af_xdp/rte_eth_af_xdp.c | 285 ++++++++++++++++++++-------- 2 files changed, 213 insertions(+), 76 deletions(-) -- 2.17.1 ^ permalink raw reply [flat|nested] 16+ messages in thread
* [dpdk-dev] [PATCH v2 1/3] net/af_xdp: enable zero copy by extbuf 2019-05-30 9:07 ` [dpdk-dev] [PATCH v2 0/3] add more features for AF_XDP pmd Xiaolong Ye @ 2019-05-30 9:07 ` Xiaolong Ye 2019-05-30 15:31 ` Stephen Hemminger 2019-06-11 16:16 ` William Tu 2019-05-30 9:07 ` [dpdk-dev] [PATCH v2 2/3] net/af_xdp: add multi-queue support Xiaolong Ye ` (2 subsequent siblings) 3 siblings, 2 replies; 16+ messages in thread From: Xiaolong Ye @ 2019-05-30 9:07 UTC (permalink / raw) To: Xiaolong Ye, Qi Zhang, John McNamara, Marko Kovacevic Cc: Karlsson Magnus, Topel Bjorn, dev Implement zero copy of af_xdp pmd through mbuf's external memory mechanism to achieve high performance. This patch also provides a new parameter "pmd_zero_copy" for user, so they can choose to enable zero copy of af_xdp pmd or not. To be clear, "zero copy" here is different from the "zero copy mode" of AF_XDP, it is about zero copy between af_xdp umem and mbuf used in dpdk application. Suggested-by: Varghese Vipin <vipin.varghese@intel.com> Suggested-by: Tummala Sivaprasad <sivaprasad.tummala@intel.com> Suggested-by: Olivier Matz <olivier.matz@6wind.com> Signed-off-by: Xiaolong Ye <xiaolong.ye@intel.com> --- doc/guides/nics/af_xdp.rst | 1 + drivers/net/af_xdp/rte_eth_af_xdp.c | 104 +++++++++++++++++++++------- 2 files changed, 79 insertions(+), 26 deletions(-) diff --git a/doc/guides/nics/af_xdp.rst b/doc/guides/nics/af_xdp.rst index 175038153..0bd4239fe 100644 --- a/doc/guides/nics/af_xdp.rst +++ b/doc/guides/nics/af_xdp.rst @@ -28,6 +28,7 @@ The following options can be provided to set up an af_xdp port in DPDK. * ``iface`` - name of the Kernel interface to attach to (required); * ``queue`` - netdev queue id (optional, default 0); +* ``pmd_zero_copy`` - enable zero copy or not (optional, default 0); Prerequisites ------------- diff --git a/drivers/net/af_xdp/rte_eth_af_xdp.c b/drivers/net/af_xdp/rte_eth_af_xdp.c index 35c72272c..014cd5691 100644 --- a/drivers/net/af_xdp/rte_eth_af_xdp.c +++ b/drivers/net/af_xdp/rte_eth_af_xdp.c @@ -70,6 +70,7 @@ struct xsk_umem_info { struct xsk_umem *umem; struct rte_ring *buf_ring; const struct rte_memzone *mz; + int pmd_zc; }; struct rx_stats { @@ -109,8 +110,8 @@ struct pmd_internals { int if_index; char if_name[IFNAMSIZ]; uint16_t queue_idx; + int pmd_zc; struct ether_addr eth_addr; - struct xsk_umem_info *umem; struct rte_mempool *mb_pool_share; struct pkt_rx_queue rx_queues[ETH_AF_XDP_MAX_QUEUE_PAIRS]; @@ -119,10 +120,12 @@ struct pmd_internals { #define ETH_AF_XDP_IFACE_ARG "iface" #define ETH_AF_XDP_QUEUE_IDX_ARG "queue" +#define ETH_AF_XDP_PMD_ZC_ARG "pmd_zero_copy" static const char * const valid_arguments[] = { ETH_AF_XDP_IFACE_ARG, ETH_AF_XDP_QUEUE_IDX_ARG, + ETH_AF_XDP_PMD_ZC_ARG, NULL }; @@ -166,6 +169,15 @@ reserve_fill_queue(struct xsk_umem_info *umem, uint16_t reserve_size) return 0; } +static void +umem_buf_release_to_fq(void *addr, void *opaque) +{ + struct xsk_umem_info *umem = (struct xsk_umem_info *)opaque; + uint64_t umem_addr = (uint64_t)addr - umem->mz->addr_64; + + rte_ring_enqueue(umem->buf_ring, (void *)umem_addr); +} + static uint16_t eth_af_xdp_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) { @@ -175,6 +187,7 @@ eth_af_xdp_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) struct xsk_ring_prod *fq = &umem->fq; uint32_t idx_rx = 0; uint32_t free_thresh = fq->size >> 1; + int pmd_zc = umem->pmd_zc; struct rte_mbuf *mbufs[ETH_AF_XDP_RX_BATCH_SIZE]; unsigned long dropped = 0; unsigned long rx_bytes = 0; @@ -197,19 +210,29 @@ eth_af_xdp_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) uint64_t addr; uint32_t len; void *pkt; + uint16_t buf_len = ETH_AF_XDP_FRAME_SIZE; + struct rte_mbuf_ext_shared_info *shinfo; desc = xsk_ring_cons__rx_desc(rx, idx_rx++); addr = desc->addr; len = desc->len; pkt = xsk_umem__get_data(rxq->umem->mz->addr, addr); - rte_memcpy(rte_pktmbuf_mtod(mbufs[i], void *), pkt, len); + if (pmd_zc) { + shinfo = rte_pktmbuf_ext_shinfo_init_helper(pkt, + &buf_len, umem_buf_release_to_fq, umem); + + rte_pktmbuf_attach_extbuf(mbufs[i], pkt, 0, buf_len, + shinfo); + } else { + rte_memcpy(rte_pktmbuf_mtod(mbufs[i], void *), + pkt, len); + rte_ring_enqueue(umem->buf_ring, (void *)addr); + } rte_pktmbuf_pkt_len(mbufs[i]) = len; rte_pktmbuf_data_len(mbufs[i]) = len; rx_bytes += len; bufs[i] = mbufs[i]; - - rte_ring_enqueue(umem->buf_ring, (void *)addr); } xsk_ring_cons__release(rx, rcvd); @@ -262,12 +285,21 @@ kick_tx(struct pkt_tx_queue *txq) pull_umem_cq(umem, ETH_AF_XDP_TX_BATCH_SIZE); } +static inline bool +in_umem_range(struct xsk_umem_info *umem, uint64_t addr) +{ + uint64_t mz_base_addr = umem->mz->addr_64; + + return addr >= mz_base_addr && addr < mz_base_addr + umem->mz->len; +} + static uint16_t eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) { struct pkt_tx_queue *txq = queue; struct xsk_umem_info *umem = txq->pair->umem; struct rte_mbuf *mbuf; + int pmd_zc = umem->pmd_zc; void *addrs[ETH_AF_XDP_TX_BATCH_SIZE]; unsigned long tx_bytes = 0; int i; @@ -294,16 +326,26 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) desc = xsk_ring_prod__tx_desc(&txq->tx, idx_tx + i); mbuf = bufs[i]; - - desc->addr = (uint64_t)addrs[i]; desc->len = mbuf->pkt_len; - pkt = xsk_umem__get_data(umem->mz->addr, - desc->addr); - rte_memcpy(pkt, rte_pktmbuf_mtod(mbuf, void *), - desc->len); - tx_bytes += mbuf->pkt_len; - rte_pktmbuf_free(mbuf); + /* + * We need to make sure the external mbuf address is within + * current port's umem memzone range + */ + if (pmd_zc && RTE_MBUF_HAS_EXTBUF(mbuf) && + in_umem_range(umem, (uint64_t)mbuf->buf_addr)) { + desc->addr = (uint64_t)mbuf->buf_addr - + umem->mz->addr_64; + mbuf->buf_addr = xsk_umem__get_data(umem->mz->addr, + (uint64_t)addrs[i]); + } else { + desc->addr = (uint64_t)addrs[i]; + pkt = xsk_umem__get_data(umem->mz->addr, + desc->addr); + rte_memcpy(pkt, rte_pktmbuf_mtod(mbuf, void *), + desc->len); + } + tx_bytes += mbuf->pkt_len; } xsk_ring_prod__submit(&txq->tx, nb_pkts); @@ -313,6 +355,9 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) txq->stats.tx_pkts += nb_pkts; txq->stats.tx_bytes += tx_bytes; + for (i = 0; i < nb_pkts; i++) + rte_pktmbuf_free(bufs[i]); + return nb_pkts; } @@ -454,18 +499,16 @@ eth_dev_close(struct rte_eth_dev *dev) if (rxq->umem == NULL) break; xsk_socket__delete(rxq->xsk); + (void)xsk_umem__delete(rxq->umem->umem); + xdp_umem_destroy(rxq->umem); } - (void)xsk_umem__delete(internals->umem->umem); - /* * MAC is not allocated dynamically, setting it to NULL would prevent * from releasing it in rte_eth_dev_release_port. */ dev->data->mac_addrs = NULL; - xdp_umem_destroy(internals->umem); - remove_xdp_program(internals); } @@ -639,7 +682,7 @@ eth_rx_queue_setup(struct rte_eth_dev *dev, goto err; } - internals->umem = rxq->umem; + rxq->umem->pmd_zc = internals->pmd_zc; dev->data->rx_queues[rx_queue_id] = rxq; return 0; @@ -775,9 +818,8 @@ parse_name_arg(const char *key __rte_unused, } static int -parse_parameters(struct rte_kvargs *kvlist, - char *if_name, - int *queue_idx) +parse_parameters(struct rte_kvargs *kvlist, char *if_name, int *queue_idx, + int *pmd_zc) { int ret; @@ -791,6 +833,11 @@ parse_parameters(struct rte_kvargs *kvlist, if (ret < 0) goto free_kvlist; + ret = rte_kvargs_process(kvlist, ETH_AF_XDP_PMD_ZC_ARG, + &parse_integer_arg, pmd_zc); + if (ret < 0) + goto free_kvlist; + free_kvlist: rte_kvargs_free(kvlist); return ret; @@ -827,9 +874,8 @@ get_iface_info(const char *if_name, } static struct rte_eth_dev * -init_internals(struct rte_vdev_device *dev, - const char *if_name, - int queue_idx) +init_internals(struct rte_vdev_device *dev, const char *if_name, int queue_idx, + int pmd_zc) { const char *name = rte_vdev_device_name(dev); const unsigned int numa_node = dev->device.numa_node; @@ -843,6 +889,7 @@ init_internals(struct rte_vdev_device *dev, return NULL; internals->queue_idx = queue_idx; + internals->pmd_zc = pmd_zc; strlcpy(internals->if_name, if_name, IFNAMSIZ); for (i = 0; i < ETH_AF_XDP_MAX_QUEUE_PAIRS; i++) { @@ -868,6 +915,9 @@ init_internals(struct rte_vdev_device *dev, /* Let rte_eth_dev_close() release the port resources. */ eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE; + if (internals->pmd_zc) + AF_XDP_LOG(INFO, "Zero copy between umem and mbuf enabled.\n"); + return eth_dev; err: @@ -883,6 +933,7 @@ rte_pmd_af_xdp_probe(struct rte_vdev_device *dev) int xsk_queue_idx = ETH_AF_XDP_DFLT_QUEUE_IDX; struct rte_eth_dev *eth_dev = NULL; const char *name; + int pmd_zc = 0; AF_XDP_LOG(INFO, "Initializing pmd_af_xdp for %s\n", rte_vdev_device_name(dev)); @@ -909,7 +960,7 @@ rte_pmd_af_xdp_probe(struct rte_vdev_device *dev) if (dev->device.numa_node == SOCKET_ID_ANY) dev->device.numa_node = rte_socket_id(); - if (parse_parameters(kvlist, if_name, &xsk_queue_idx) < 0) { + if (parse_parameters(kvlist, if_name, &xsk_queue_idx, &pmd_zc) < 0) { AF_XDP_LOG(ERR, "Invalid kvargs value\n"); return -EINVAL; } @@ -919,7 +970,7 @@ rte_pmd_af_xdp_probe(struct rte_vdev_device *dev) return -EINVAL; } - eth_dev = init_internals(dev, if_name, xsk_queue_idx); + eth_dev = init_internals(dev, if_name, xsk_queue_idx, pmd_zc); if (eth_dev == NULL) { AF_XDP_LOG(ERR, "Failed to init internals\n"); return -1; @@ -961,7 +1012,8 @@ static struct rte_vdev_driver pmd_af_xdp_drv = { RTE_PMD_REGISTER_VDEV(net_af_xdp, pmd_af_xdp_drv); RTE_PMD_REGISTER_PARAM_STRING(net_af_xdp, "iface=<string> " - "queue=<int> "); + "queue=<int> " + "pmd_zero_copy=<0|1>"); RTE_INIT(af_xdp_init_log) { -- 2.17.1 ^ permalink raw reply related [flat|nested] 16+ messages in thread
* Re: [dpdk-dev] [PATCH v2 1/3] net/af_xdp: enable zero copy by extbuf 2019-05-30 9:07 ` [dpdk-dev] [PATCH v2 1/3] net/af_xdp: enable zero copy by extbuf Xiaolong Ye @ 2019-05-30 15:31 ` Stephen Hemminger 2019-05-31 1:49 ` Ye Xiaolong 2019-06-11 16:16 ` William Tu 1 sibling, 1 reply; 16+ messages in thread From: Stephen Hemminger @ 2019-05-30 15:31 UTC (permalink / raw) To: Xiaolong Ye Cc: Qi Zhang, John McNamara, Marko Kovacevic, Karlsson Magnus, Topel Bjorn, dev On Thu, 30 May 2019 17:07:05 +0800 Xiaolong Ye <xiaolong.ye@intel.com> wrote: > Implement zero copy of af_xdp pmd through mbuf's external memory > mechanism to achieve high performance. > > This patch also provides a new parameter "pmd_zero_copy" for user, so they > can choose to enable zero copy of af_xdp pmd or not. > > To be clear, "zero copy" here is different from the "zero copy mode" of > AF_XDP, it is about zero copy between af_xdp umem and mbuf used in dpdk > application. > > Suggested-by: Varghese Vipin <vipin.varghese@intel.com> > Suggested-by: Tummala Sivaprasad <sivaprasad.tummala@intel.com> > Suggested-by: Olivier Matz <olivier.matz@6wind.com> > Signed-off-by: Xiaolong Ye <xiaolong.ye@intel.com> Why is this a parameter? Can it just be auto detected. Remember configuration is evil, it hurts usability, code coverage and increases complexity. ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [dpdk-dev] [PATCH v2 1/3] net/af_xdp: enable zero copy by extbuf 2019-05-30 15:31 ` Stephen Hemminger @ 2019-05-31 1:49 ` Ye Xiaolong 0 siblings, 0 replies; 16+ messages in thread From: Ye Xiaolong @ 2019-05-31 1:49 UTC (permalink / raw) To: Stephen Hemminger Cc: Qi Zhang, John McNamara, Marko Kovacevic, Karlsson Magnus, Topel Bjorn, Ananyev Konstantin, dev On 05/30, Stephen Hemminger wrote: >On Thu, 30 May 2019 17:07:05 +0800 >Xiaolong Ye <xiaolong.ye@intel.com> wrote: > >> Implement zero copy of af_xdp pmd through mbuf's external memory >> mechanism to achieve high performance. >> >> This patch also provides a new parameter "pmd_zero_copy" for user, so they >> can choose to enable zero copy of af_xdp pmd or not. >> >> To be clear, "zero copy" here is different from the "zero copy mode" of >> AF_XDP, it is about zero copy between af_xdp umem and mbuf used in dpdk >> application. >> >> Suggested-by: Varghese Vipin <vipin.varghese@intel.com> >> Suggested-by: Tummala Sivaprasad <sivaprasad.tummala@intel.com> >> Suggested-by: Olivier Matz <olivier.matz@6wind.com> >> Signed-off-by: Xiaolong Ye <xiaolong.ye@intel.com> > >Why is this a parameter? Can it just be auto detected. In case there might be situations when some users would perfer a slower copy mode, we can just let user decide. Thanks, Xiaolong >Remember configuration is evil, it hurts usability, code coverage >and increases complexity. ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [dpdk-dev] [PATCH v2 1/3] net/af_xdp: enable zero copy by extbuf 2019-05-30 9:07 ` [dpdk-dev] [PATCH v2 1/3] net/af_xdp: enable zero copy by extbuf Xiaolong Ye 2019-05-30 15:31 ` Stephen Hemminger @ 2019-06-11 16:16 ` William Tu 2019-06-12 10:03 ` Ye Xiaolong 1 sibling, 1 reply; 16+ messages in thread From: William Tu @ 2019-06-11 16:16 UTC (permalink / raw) To: Xiaolong Ye Cc: Qi Zhang, John McNamara, Marko Kovacevic, Karlsson Magnus, Topel Bjorn, dev On Thu, May 30, 2019 at 2:16 AM Xiaolong Ye <xiaolong.ye@intel.com> wrote: > > Implement zero copy of af_xdp pmd through mbuf's external memory > mechanism to achieve high performance. > > This patch also provides a new parameter "pmd_zero_copy" for user, so they > can choose to enable zero copy of af_xdp pmd or not. > > To be clear, "zero copy" here is different from the "zero copy mode" of > AF_XDP, it is about zero copy between af_xdp umem and mbuf used in dpdk > application. > > Suggested-by: Varghese Vipin <vipin.varghese@intel.com> > Suggested-by: Tummala Sivaprasad <sivaprasad.tummala@intel.com> > Suggested-by: Olivier Matz <olivier.matz@6wind.com> > Signed-off-by: Xiaolong Ye <xiaolong.ye@intel.com> > --- > doc/guides/nics/af_xdp.rst | 1 + > drivers/net/af_xdp/rte_eth_af_xdp.c | 104 +++++++++++++++++++++------- > 2 files changed, 79 insertions(+), 26 deletions(-) > > diff --git a/doc/guides/nics/af_xdp.rst b/doc/guides/nics/af_xdp.rst > index 175038153..0bd4239fe 100644 > --- a/doc/guides/nics/af_xdp.rst > +++ b/doc/guides/nics/af_xdp.rst > @@ -28,6 +28,7 @@ The following options can be provided to set up an af_xdp port in DPDK. > > * ``iface`` - name of the Kernel interface to attach to (required); > * ``queue`` - netdev queue id (optional, default 0); > +* ``pmd_zero_copy`` - enable zero copy or not (optional, default 0); > > Prerequisites > ------------- > diff --git a/drivers/net/af_xdp/rte_eth_af_xdp.c b/drivers/net/af_xdp/rte_eth_af_xdp.c > index 35c72272c..014cd5691 100644 > --- a/drivers/net/af_xdp/rte_eth_af_xdp.c > +++ b/drivers/net/af_xdp/rte_eth_af_xdp.c > @@ -70,6 +70,7 @@ struct xsk_umem_info { > struct xsk_umem *umem; > struct rte_ring *buf_ring; > const struct rte_memzone *mz; > + int pmd_zc; > }; > > struct rx_stats { > @@ -109,8 +110,8 @@ struct pmd_internals { > int if_index; > char if_name[IFNAMSIZ]; > uint16_t queue_idx; > + int pmd_zc; > struct ether_addr eth_addr; > - struct xsk_umem_info *umem; > struct rte_mempool *mb_pool_share; > > struct pkt_rx_queue rx_queues[ETH_AF_XDP_MAX_QUEUE_PAIRS]; > @@ -119,10 +120,12 @@ struct pmd_internals { > > #define ETH_AF_XDP_IFACE_ARG "iface" > #define ETH_AF_XDP_QUEUE_IDX_ARG "queue" > +#define ETH_AF_XDP_PMD_ZC_ARG "pmd_zero_copy" > > static const char * const valid_arguments[] = { > ETH_AF_XDP_IFACE_ARG, > ETH_AF_XDP_QUEUE_IDX_ARG, > + ETH_AF_XDP_PMD_ZC_ARG, > NULL > }; > > @@ -166,6 +169,15 @@ reserve_fill_queue(struct xsk_umem_info *umem, uint16_t reserve_size) > return 0; > } > > +static void > +umem_buf_release_to_fq(void *addr, void *opaque) > +{ > + struct xsk_umem_info *umem = (struct xsk_umem_info *)opaque; > + uint64_t umem_addr = (uint64_t)addr - umem->mz->addr_64; > + > + rte_ring_enqueue(umem->buf_ring, (void *)umem_addr); > +} > + > static uint16_t > eth_af_xdp_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) > { > @@ -175,6 +187,7 @@ eth_af_xdp_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) > struct xsk_ring_prod *fq = &umem->fq; > uint32_t idx_rx = 0; > uint32_t free_thresh = fq->size >> 1; > + int pmd_zc = umem->pmd_zc; > struct rte_mbuf *mbufs[ETH_AF_XDP_RX_BATCH_SIZE]; > unsigned long dropped = 0; > unsigned long rx_bytes = 0; > @@ -197,19 +210,29 @@ eth_af_xdp_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) > uint64_t addr; > uint32_t len; > void *pkt; > + uint16_t buf_len = ETH_AF_XDP_FRAME_SIZE; > + struct rte_mbuf_ext_shared_info *shinfo; > > desc = xsk_ring_cons__rx_desc(rx, idx_rx++); > addr = desc->addr; > len = desc->len; > pkt = xsk_umem__get_data(rxq->umem->mz->addr, addr); > > - rte_memcpy(rte_pktmbuf_mtod(mbufs[i], void *), pkt, len); > + if (pmd_zc) { > + shinfo = rte_pktmbuf_ext_shinfo_init_helper(pkt, > + &buf_len, umem_buf_release_to_fq, umem); > + > + rte_pktmbuf_attach_extbuf(mbufs[i], pkt, 0, buf_len, > + shinfo); > + } else { > + rte_memcpy(rte_pktmbuf_mtod(mbufs[i], void *), > + pkt, len); > + rte_ring_enqueue(umem->buf_ring, (void *)addr); > + } > rte_pktmbuf_pkt_len(mbufs[i]) = len; > rte_pktmbuf_data_len(mbufs[i]) = len; > rx_bytes += len; > bufs[i] = mbufs[i]; > - > - rte_ring_enqueue(umem->buf_ring, (void *)addr); > } > > xsk_ring_cons__release(rx, rcvd); > @@ -262,12 +285,21 @@ kick_tx(struct pkt_tx_queue *txq) > pull_umem_cq(umem, ETH_AF_XDP_TX_BATCH_SIZE); > } > > +static inline bool > +in_umem_range(struct xsk_umem_info *umem, uint64_t addr) > +{ > + uint64_t mz_base_addr = umem->mz->addr_64; > + > + return addr >= mz_base_addr && addr < mz_base_addr + umem->mz->len; > +} > + > static uint16_t > eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) > { > struct pkt_tx_queue *txq = queue; > struct xsk_umem_info *umem = txq->pair->umem; > struct rte_mbuf *mbuf; > + int pmd_zc = umem->pmd_zc; > void *addrs[ETH_AF_XDP_TX_BATCH_SIZE]; > unsigned long tx_bytes = 0; > int i; > @@ -294,16 +326,26 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) > > desc = xsk_ring_prod__tx_desc(&txq->tx, idx_tx + i); > mbuf = bufs[i]; > - > - desc->addr = (uint64_t)addrs[i]; > desc->len = mbuf->pkt_len; > - pkt = xsk_umem__get_data(umem->mz->addr, > - desc->addr); > - rte_memcpy(pkt, rte_pktmbuf_mtod(mbuf, void *), > - desc->len); > - tx_bytes += mbuf->pkt_len; > > - rte_pktmbuf_free(mbuf); > + /* > + * We need to make sure the external mbuf address is within > + * current port's umem memzone range > + */ > + if (pmd_zc && RTE_MBUF_HAS_EXTBUF(mbuf) && > + in_umem_range(umem, (uint64_t)mbuf->buf_addr)) { > + desc->addr = (uint64_t)mbuf->buf_addr - > + umem->mz->addr_64; > + mbuf->buf_addr = xsk_umem__get_data(umem->mz->addr, > + (uint64_t)addrs[i]); > + } else { > + desc->addr = (uint64_t)addrs[i]; > + pkt = xsk_umem__get_data(umem->mz->addr, > + desc->addr); > + rte_memcpy(pkt, rte_pktmbuf_mtod(mbuf, void *), > + desc->len); > + } > + tx_bytes += mbuf->pkt_len; > } > > xsk_ring_prod__submit(&txq->tx, nb_pkts); > @@ -313,6 +355,9 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) > txq->stats.tx_pkts += nb_pkts; > txq->stats.tx_bytes += tx_bytes; > > + for (i = 0; i < nb_pkts; i++) > + rte_pktmbuf_free(bufs[i]); > + Is it ok to free the mbuf here? If the AF_XDP is running pmd_zc=true, the packet mbuf is still in the tx ring and might not be sent out yet. Regards, William > return nb_pkts; > } > > @@ -454,18 +499,16 @@ eth_dev_close(struct rte_eth_dev *dev) > if (rxq->umem == NULL) > break; > xsk_socket__delete(rxq->xsk); > + (void)xsk_umem__delete(rxq->umem->umem); > + xdp_umem_destroy(rxq->umem); > } > > - (void)xsk_umem__delete(internals->umem->umem); > - > /* > * MAC is not allocated dynamically, setting it to NULL would prevent > * from releasing it in rte_eth_dev_release_port. > */ > dev->data->mac_addrs = NULL; > > - xdp_umem_destroy(internals->umem); > - > remove_xdp_program(internals); > } > > @@ -639,7 +682,7 @@ eth_rx_queue_setup(struct rte_eth_dev *dev, > goto err; > } > > - internals->umem = rxq->umem; > + rxq->umem->pmd_zc = internals->pmd_zc; > > dev->data->rx_queues[rx_queue_id] = rxq; > return 0; > @@ -775,9 +818,8 @@ parse_name_arg(const char *key __rte_unused, > } > > static int > -parse_parameters(struct rte_kvargs *kvlist, > - char *if_name, > - int *queue_idx) > +parse_parameters(struct rte_kvargs *kvlist, char *if_name, int *queue_idx, > + int *pmd_zc) > { > int ret; > > @@ -791,6 +833,11 @@ parse_parameters(struct rte_kvargs *kvlist, > if (ret < 0) > goto free_kvlist; > > + ret = rte_kvargs_process(kvlist, ETH_AF_XDP_PMD_ZC_ARG, > + &parse_integer_arg, pmd_zc); > + if (ret < 0) > + goto free_kvlist; > + > free_kvlist: > rte_kvargs_free(kvlist); > return ret; > @@ -827,9 +874,8 @@ get_iface_info(const char *if_name, > } > > static struct rte_eth_dev * > -init_internals(struct rte_vdev_device *dev, > - const char *if_name, > - int queue_idx) > +init_internals(struct rte_vdev_device *dev, const char *if_name, int queue_idx, > + int pmd_zc) > { > const char *name = rte_vdev_device_name(dev); > const unsigned int numa_node = dev->device.numa_node; > @@ -843,6 +889,7 @@ init_internals(struct rte_vdev_device *dev, > return NULL; > > internals->queue_idx = queue_idx; > + internals->pmd_zc = pmd_zc; > strlcpy(internals->if_name, if_name, IFNAMSIZ); > > for (i = 0; i < ETH_AF_XDP_MAX_QUEUE_PAIRS; i++) { > @@ -868,6 +915,9 @@ init_internals(struct rte_vdev_device *dev, > /* Let rte_eth_dev_close() release the port resources. */ > eth_dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE; > > + if (internals->pmd_zc) > + AF_XDP_LOG(INFO, "Zero copy between umem and mbuf enabled.\n"); > + > return eth_dev; > > err: > @@ -883,6 +933,7 @@ rte_pmd_af_xdp_probe(struct rte_vdev_device *dev) > int xsk_queue_idx = ETH_AF_XDP_DFLT_QUEUE_IDX; > struct rte_eth_dev *eth_dev = NULL; > const char *name; > + int pmd_zc = 0; > > AF_XDP_LOG(INFO, "Initializing pmd_af_xdp for %s\n", > rte_vdev_device_name(dev)); > @@ -909,7 +960,7 @@ rte_pmd_af_xdp_probe(struct rte_vdev_device *dev) > if (dev->device.numa_node == SOCKET_ID_ANY) > dev->device.numa_node = rte_socket_id(); > > - if (parse_parameters(kvlist, if_name, &xsk_queue_idx) < 0) { > + if (parse_parameters(kvlist, if_name, &xsk_queue_idx, &pmd_zc) < 0) { > AF_XDP_LOG(ERR, "Invalid kvargs value\n"); > return -EINVAL; > } > @@ -919,7 +970,7 @@ rte_pmd_af_xdp_probe(struct rte_vdev_device *dev) > return -EINVAL; > } > > - eth_dev = init_internals(dev, if_name, xsk_queue_idx); > + eth_dev = init_internals(dev, if_name, xsk_queue_idx, pmd_zc); > if (eth_dev == NULL) { > AF_XDP_LOG(ERR, "Failed to init internals\n"); > return -1; > @@ -961,7 +1012,8 @@ static struct rte_vdev_driver pmd_af_xdp_drv = { > RTE_PMD_REGISTER_VDEV(net_af_xdp, pmd_af_xdp_drv); > RTE_PMD_REGISTER_PARAM_STRING(net_af_xdp, > "iface=<string> " > - "queue=<int> "); > + "queue=<int> " > + "pmd_zero_copy=<0|1>"); > > RTE_INIT(af_xdp_init_log) > { > -- > 2.17.1 > ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [dpdk-dev] [PATCH v2 1/3] net/af_xdp: enable zero copy by extbuf 2019-06-11 16:16 ` William Tu @ 2019-06-12 10:03 ` Ye Xiaolong 2019-06-13 0:32 ` William Tu 0 siblings, 1 reply; 16+ messages in thread From: Ye Xiaolong @ 2019-06-12 10:03 UTC (permalink / raw) To: William Tu Cc: Qi Zhang, John McNamara, Marko Kovacevic, Karlsson Magnus, Topel Bjorn, dev Hi, On 06/11, William Tu wrote: [snip] >> @@ -294,16 +326,26 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) >> >> desc = xsk_ring_prod__tx_desc(&txq->tx, idx_tx + i); >> mbuf = bufs[i]; >> - >> - desc->addr = (uint64_t)addrs[i]; >> desc->len = mbuf->pkt_len; >> - pkt = xsk_umem__get_data(umem->mz->addr, >> - desc->addr); >> - rte_memcpy(pkt, rte_pktmbuf_mtod(mbuf, void *), >> - desc->len); >> - tx_bytes += mbuf->pkt_len; >> >> - rte_pktmbuf_free(mbuf); >> + /* >> + * We need to make sure the external mbuf address is within >> + * current port's umem memzone range >> + */ >> + if (pmd_zc && RTE_MBUF_HAS_EXTBUF(mbuf) && >> + in_umem_range(umem, (uint64_t)mbuf->buf_addr)) { >> + desc->addr = (uint64_t)mbuf->buf_addr - >> + umem->mz->addr_64; >> + mbuf->buf_addr = xsk_umem__get_data(umem->mz->addr, >> + (uint64_t)addrs[i]); >> + } else { >> + desc->addr = (uint64_t)addrs[i]; >> + pkt = xsk_umem__get_data(umem->mz->addr, >> + desc->addr); >> + rte_memcpy(pkt, rte_pktmbuf_mtod(mbuf, void *), >> + desc->len); >> + } >> + tx_bytes += mbuf->pkt_len; >> } >> >> xsk_ring_prod__submit(&txq->tx, nb_pkts); >> @@ -313,6 +355,9 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) >> txq->stats.tx_pkts += nb_pkts; >> txq->stats.tx_bytes += tx_bytes; >> >> + for (i = 0; i < nb_pkts; i++) >> + rte_pktmbuf_free(bufs[i]); >> + > >Is it ok to free the mbuf here? >If the AF_XDP is running pmd_zc=true, the packet mbuf is still in the tx >ring and might not be sent out yet. For pmd_zc=ture case, here mbuf->buf_addr has been exchanged to available addr dequeued from umem->buf_ring, rte_pktmbuf_free would just call the free callback umem_buf_release_to_fq to enqueue the addr to the buf_ring. Thanks, Xiaolong > >Regards, >William > ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [dpdk-dev] [PATCH v2 1/3] net/af_xdp: enable zero copy by extbuf 2019-06-12 10:03 ` Ye Xiaolong @ 2019-06-13 0:32 ` William Tu 0 siblings, 0 replies; 16+ messages in thread From: William Tu @ 2019-06-13 0:32 UTC (permalink / raw) To: Ye Xiaolong Cc: Qi Zhang, John McNamara, Marko Kovacevic, Karlsson Magnus, Topel Bjorn, dev On Tue, Jun 11, 2019 at 8:21 PM Ye Xiaolong <xiaolong.ye@intel.com> wrote: > > Hi, > > On 06/11, William Tu wrote: > [snip] > >> @@ -294,16 +326,26 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) > >> > >> desc = xsk_ring_prod__tx_desc(&txq->tx, idx_tx + i); > >> mbuf = bufs[i]; > >> - > >> - desc->addr = (uint64_t)addrs[i]; > >> desc->len = mbuf->pkt_len; > >> - pkt = xsk_umem__get_data(umem->mz->addr, > >> - desc->addr); > >> - rte_memcpy(pkt, rte_pktmbuf_mtod(mbuf, void *), > >> - desc->len); > >> - tx_bytes += mbuf->pkt_len; > >> > >> - rte_pktmbuf_free(mbuf); > >> + /* > >> + * We need to make sure the external mbuf address is within > >> + * current port's umem memzone range > >> + */ > >> + if (pmd_zc && RTE_MBUF_HAS_EXTBUF(mbuf) && > >> + in_umem_range(umem, (uint64_t)mbuf->buf_addr)) { > >> + desc->addr = (uint64_t)mbuf->buf_addr - > >> + umem->mz->addr_64; > >> + mbuf->buf_addr = xsk_umem__get_data(umem->mz->addr, > >> + (uint64_t)addrs[i]); > >> + } else { > >> + desc->addr = (uint64_t)addrs[i]; > >> + pkt = xsk_umem__get_data(umem->mz->addr, > >> + desc->addr); > >> + rte_memcpy(pkt, rte_pktmbuf_mtod(mbuf, void *), > >> + desc->len); > >> + } > >> + tx_bytes += mbuf->pkt_len; > >> } > >> > >> xsk_ring_prod__submit(&txq->tx, nb_pkts); > >> @@ -313,6 +355,9 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) > >> txq->stats.tx_pkts += nb_pkts; > >> txq->stats.tx_bytes += tx_bytes; > >> > >> + for (i = 0; i < nb_pkts; i++) > >> + rte_pktmbuf_free(bufs[i]); > >> + > > > >Is it ok to free the mbuf here? > >If the AF_XDP is running pmd_zc=true, the packet mbuf is still in the tx > >ring and might not be sent out yet. > > For pmd_zc=ture case, here mbuf->buf_addr has been exchanged to available addr > dequeued from umem->buf_ring, rte_pktmbuf_free would just call the free callback > umem_buf_release_to_fq to enqueue the addr to the buf_ring. > I see, thanks for the explanation. William > Thanks, > Xiaolong > > > > >Regards, > >William > > ^ permalink raw reply [flat|nested] 16+ messages in thread
* [dpdk-dev] [PATCH v2 2/3] net/af_xdp: add multi-queue support 2019-05-30 9:07 ` [dpdk-dev] [PATCH v2 0/3] add more features for AF_XDP pmd Xiaolong Ye 2019-05-30 9:07 ` [dpdk-dev] [PATCH v2 1/3] net/af_xdp: enable zero copy by extbuf Xiaolong Ye @ 2019-05-30 9:07 ` Xiaolong Ye 2019-05-30 15:32 ` Stephen Hemminger 2019-05-30 9:07 ` [dpdk-dev] [PATCH v2 3/3] net/af_xdp: remove unused struct member Xiaolong Ye 2019-06-10 16:54 ` [dpdk-dev] [PATCH v2 0/3] add more features for AF_XDP pmd Ferruh Yigit 3 siblings, 1 reply; 16+ messages in thread From: Xiaolong Ye @ 2019-05-30 9:07 UTC (permalink / raw) To: Xiaolong Ye, Qi Zhang, John McNamara, Marko Kovacevic Cc: Karlsson Magnus, Topel Bjorn, dev This patch adds two parameters `start_queue` and `queue_count` to specify the range of netdev queues used by AF_XDP pmd. Signed-off-by: Xiaolong Ye <xiaolong.ye@intel.com> --- doc/guides/nics/af_xdp.rst | 3 +- drivers/net/af_xdp/rte_eth_af_xdp.c | 194 ++++++++++++++++++++-------- 2 files changed, 141 insertions(+), 56 deletions(-) diff --git a/doc/guides/nics/af_xdp.rst b/doc/guides/nics/af_xdp.rst index 0bd4239fe..18defcda3 100644 --- a/doc/guides/nics/af_xdp.rst +++ b/doc/guides/nics/af_xdp.rst @@ -27,7 +27,8 @@ Options The following options can be provided to set up an af_xdp port in DPDK. * ``iface`` - name of the Kernel interface to attach to (required); -* ``queue`` - netdev queue id (optional, default 0); +* ``start_queue`` - starting netdev queue id (optional, default 0); +* ``queue_count`` - total netdev queue number (optional, default 1); * ``pmd_zero_copy`` - enable zero copy or not (optional, default 0); Prerequisites diff --git a/drivers/net/af_xdp/rte_eth_af_xdp.c b/drivers/net/af_xdp/rte_eth_af_xdp.c index 014cd5691..f56aabcae 100644 --- a/drivers/net/af_xdp/rte_eth_af_xdp.c +++ b/drivers/net/af_xdp/rte_eth_af_xdp.c @@ -12,6 +12,8 @@ #include <linux/if_ether.h> #include <linux/if_xdp.h> #include <linux/if_link.h> +#include <linux/ethtool.h> +#include <linux/sockios.h> #include "af_xdp_deps.h" #include <bpf/xsk.h> @@ -57,12 +59,12 @@ static int af_xdp_logtype; #define ETH_AF_XDP_NUM_BUFFERS 4096 #define ETH_AF_XDP_DATA_HEADROOM 0 #define ETH_AF_XDP_DFLT_NUM_DESCS XSK_RING_CONS__DEFAULT_NUM_DESCS -#define ETH_AF_XDP_DFLT_QUEUE_IDX 0 +#define ETH_AF_XDP_DFLT_START_QUEUE_IDX 0 +#define ETH_AF_XDP_DFLT_QUEUE_COUNT 1 #define ETH_AF_XDP_RX_BATCH_SIZE 32 #define ETH_AF_XDP_TX_BATCH_SIZE 32 -#define ETH_AF_XDP_MAX_QUEUE_PAIRS 16 struct xsk_umem_info { struct xsk_ring_prod fq; @@ -88,7 +90,7 @@ struct pkt_rx_queue { struct rx_stats stats; struct pkt_tx_queue *pair; - uint16_t queue_idx; + int xsk_queue_idx; }; struct tx_stats { @@ -103,28 +105,34 @@ struct pkt_tx_queue { struct tx_stats stats; struct pkt_rx_queue *pair; - uint16_t queue_idx; + int xsk_queue_idx; }; struct pmd_internals { int if_index; char if_name[IFNAMSIZ]; - uint16_t queue_idx; + int start_queue_idx; + int queue_cnt; + int max_queue_cnt; + int combined_queue_cnt; + int pmd_zc; struct ether_addr eth_addr; struct rte_mempool *mb_pool_share; - struct pkt_rx_queue rx_queues[ETH_AF_XDP_MAX_QUEUE_PAIRS]; - struct pkt_tx_queue tx_queues[ETH_AF_XDP_MAX_QUEUE_PAIRS]; + struct pkt_rx_queue *rx_queues; + struct pkt_tx_queue *tx_queues; }; #define ETH_AF_XDP_IFACE_ARG "iface" -#define ETH_AF_XDP_QUEUE_IDX_ARG "queue" +#define ETH_AF_XDP_START_QUEUE_ARG "start_queue" +#define ETH_AF_XDP_QUEUE_COUNT_ARG "queue_count" #define ETH_AF_XDP_PMD_ZC_ARG "pmd_zero_copy" static const char * const valid_arguments[] = { ETH_AF_XDP_IFACE_ARG, - ETH_AF_XDP_QUEUE_IDX_ARG, + ETH_AF_XDP_START_QUEUE_ARG, + ETH_AF_XDP_QUEUE_COUNT_ARG, ETH_AF_XDP_PMD_ZC_ARG, NULL }; @@ -394,8 +402,8 @@ eth_dev_info(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info) dev_info->if_index = internals->if_index; dev_info->max_mac_addrs = 1; dev_info->max_rx_pktlen = ETH_FRAME_LEN; - dev_info->max_rx_queues = 1; - dev_info->max_tx_queues = 1; + dev_info->max_rx_queues = internals->queue_cnt; + dev_info->max_tx_queues = internals->queue_cnt; dev_info->min_mtu = ETHER_MIN_MTU; dev_info->max_mtu = ETH_AF_XDP_FRAME_SIZE - ETH_AF_XDP_DATA_HEADROOM; @@ -412,21 +420,23 @@ eth_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats) struct pmd_internals *internals = dev->data->dev_private; struct xdp_statistics xdp_stats; struct pkt_rx_queue *rxq; + struct pkt_tx_queue *txq; socklen_t optlen; int i, ret; for (i = 0; i < dev->data->nb_rx_queues; i++) { optlen = sizeof(struct xdp_statistics); rxq = &internals->rx_queues[i]; - stats->q_ipackets[i] = internals->rx_queues[i].stats.rx_pkts; - stats->q_ibytes[i] = internals->rx_queues[i].stats.rx_bytes; + txq = rxq->pair; + stats->q_ipackets[i] = rxq->stats.rx_pkts; + stats->q_ibytes[i] = rxq->stats.rx_bytes; - stats->q_opackets[i] = internals->tx_queues[i].stats.tx_pkts; - stats->q_obytes[i] = internals->tx_queues[i].stats.tx_bytes; + stats->q_opackets[i] = txq->stats.tx_pkts; + stats->q_obytes[i] = txq->stats.tx_bytes; stats->ipackets += stats->q_ipackets[i]; stats->ibytes += stats->q_ibytes[i]; - stats->imissed += internals->rx_queues[i].stats.rx_dropped; + stats->imissed += rxq->stats.rx_dropped; ret = getsockopt(xsk_socket__fd(rxq->xsk), SOL_XDP, XDP_STATISTICS, &xdp_stats, &optlen); if (ret != 0) { @@ -436,7 +446,7 @@ eth_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats) stats->imissed += xdp_stats.rx_dropped; stats->opackets += stats->q_opackets[i]; - stats->oerrors += internals->tx_queues[i].stats.err_pkts; + stats->oerrors += txq->stats.err_pkts; stats->obytes += stats->q_obytes[i]; } @@ -449,7 +459,7 @@ eth_stats_reset(struct rte_eth_dev *dev) struct pmd_internals *internals = dev->data->dev_private; int i; - for (i = 0; i < ETH_AF_XDP_MAX_QUEUE_PAIRS; i++) { + for (i = 0; i < internals->queue_cnt; i++) { memset(&internals->rx_queues[i].stats, 0, sizeof(struct rx_stats)); memset(&internals->tx_queues[i].stats, 0, @@ -494,13 +504,17 @@ eth_dev_close(struct rte_eth_dev *dev) AF_XDP_LOG(INFO, "Closing AF_XDP ethdev on numa socket %u\n", rte_socket_id()); - for (i = 0; i < ETH_AF_XDP_MAX_QUEUE_PAIRS; i++) { + for (i = 0; i < internals->queue_cnt; i++) { rxq = &internals->rx_queues[i]; if (rxq->umem == NULL) break; xsk_socket__delete(rxq->xsk); (void)xsk_umem__delete(rxq->umem->umem); xdp_umem_destroy(rxq->umem); + + /* free pkt_tx_queue */ + rte_free(rxq->pair); + rte_free(rxq); } /* @@ -525,7 +539,8 @@ eth_link_update(struct rte_eth_dev *dev __rte_unused, } static struct -xsk_umem_info *xdp_umem_configure(struct pmd_internals *internals) +xsk_umem_info *xdp_umem_configure(struct pmd_internals *internals, + struct pkt_rx_queue *rxq) { struct xsk_umem_info *umem; const struct rte_memzone *mz; @@ -546,7 +561,7 @@ xsk_umem_info *xdp_umem_configure(struct pmd_internals *internals) } snprintf(ring_name, sizeof(ring_name), "af_xdp_ring_%s_%u", - internals->if_name, internals->queue_idx); + internals->if_name, rxq->xsk_queue_idx); umem->buf_ring = rte_ring_create(ring_name, ETH_AF_XDP_NUM_BUFFERS, rte_socket_id(), @@ -562,7 +577,7 @@ xsk_umem_info *xdp_umem_configure(struct pmd_internals *internals) ETH_AF_XDP_DATA_HEADROOM)); snprintf(mz_name, sizeof(mz_name), "af_xdp_umem_%s_%u", - internals->if_name, internals->queue_idx); + internals->if_name, rxq->xsk_queue_idx); mz = rte_memzone_reserve_aligned(mz_name, ETH_AF_XDP_NUM_BUFFERS * ETH_AF_XDP_FRAME_SIZE, rte_socket_id(), RTE_MEMZONE_IOVA_CONTIG, @@ -599,7 +614,7 @@ xsk_configure(struct pmd_internals *internals, struct pkt_rx_queue *rxq, int ret = 0; int reserve_size; - rxq->umem = xdp_umem_configure(internals); + rxq->umem = xdp_umem_configure(internals, rxq); if (rxq->umem == NULL) return -ENOMEM; @@ -609,7 +624,7 @@ xsk_configure(struct pmd_internals *internals, struct pkt_rx_queue *rxq, cfg.xdp_flags = XDP_FLAGS_UPDATE_IF_NOEXIST; cfg.bind_flags = 0; ret = xsk_socket__create(&rxq->xsk, internals->if_name, - internals->queue_idx, rxq->umem->umem, &rxq->rx, + rxq->xsk_queue_idx, rxq->umem->umem, &rxq->rx, &txq->tx, &cfg); if (ret) { AF_XDP_LOG(ERR, "Failed to create xsk socket.\n"); @@ -632,20 +647,6 @@ xsk_configure(struct pmd_internals *internals, struct pkt_rx_queue *rxq, return ret; } -static void -queue_reset(struct pmd_internals *internals, uint16_t queue_idx) -{ - struct pkt_rx_queue *rxq = &internals->rx_queues[queue_idx]; - struct pkt_tx_queue *txq = rxq->pair; - - memset(rxq, 0, sizeof(*rxq)); - memset(txq, 0, sizeof(*txq)); - rxq->pair = txq; - txq->pair = rxq; - rxq->queue_idx = queue_idx; - txq->queue_idx = queue_idx; -} - static int eth_rx_queue_setup(struct rte_eth_dev *dev, uint16_t rx_queue_id, @@ -660,8 +661,9 @@ eth_rx_queue_setup(struct rte_eth_dev *dev, int ret; rxq = &internals->rx_queues[rx_queue_id]; - queue_reset(internals, rx_queue_id); + AF_XDP_LOG(INFO, "Set up rx queue, rx queue id: %d, xsk queue id: %d\n", + rx_queue_id, rxq->xsk_queue_idx); /* Now get the space available for data in the mbuf */ buf_size = rte_pktmbuf_data_room_size(mb_pool) - RTE_PKTMBUF_HEADROOM; @@ -688,7 +690,6 @@ eth_rx_queue_setup(struct rte_eth_dev *dev, return 0; err: - queue_reset(internals, rx_queue_id); return ret; } @@ -818,8 +819,45 @@ parse_name_arg(const char *key __rte_unused, } static int -parse_parameters(struct rte_kvargs *kvlist, char *if_name, int *queue_idx, - int *pmd_zc) +xdp_get_channels_info(const char *if_name, int *max_queues, + int *combined_queues) +{ + struct ethtool_channels channels; + struct ifreq ifr; + int fd, ret; + + fd = socket(AF_INET, SOCK_DGRAM, 0); + if (fd < 0) + return -1; + + channels.cmd = ETHTOOL_GCHANNELS; + ifr.ifr_data = (void *)&channels; + strncpy(ifr.ifr_name, if_name, IFNAMSIZ); + ret = ioctl(fd, SIOCETHTOOL, &ifr); + if (ret && errno != EOPNOTSUPP) { + ret = -errno; + goto out; + } + + if (channels.max_combined == 0 || errno == EOPNOTSUPP) { + /* If the device says it has no channels, then all traffic + * is sent to a single stream, so max queues = 1. + */ + *max_queues = 1; + *combined_queues = 1; + } else { + *max_queues = channels.max_combined; + *combined_queues = channels.combined_count; + } + + out: + close(fd); + return ret; +} + +static int +parse_parameters(struct rte_kvargs *kvlist, char *if_name, int *start_queue, + int *queue_cnt, int *pmd_zc) { int ret; @@ -828,11 +866,18 @@ parse_parameters(struct rte_kvargs *kvlist, char *if_name, int *queue_idx, if (ret < 0) goto free_kvlist; - ret = rte_kvargs_process(kvlist, ETH_AF_XDP_QUEUE_IDX_ARG, - &parse_integer_arg, queue_idx); + ret = rte_kvargs_process(kvlist, ETH_AF_XDP_START_QUEUE_ARG, + &parse_integer_arg, start_queue); if (ret < 0) goto free_kvlist; + ret = rte_kvargs_process(kvlist, ETH_AF_XDP_QUEUE_COUNT_ARG, + &parse_integer_arg, queue_cnt); + if (ret < 0 || *queue_cnt <= 0) { + ret = -EINVAL; + goto free_kvlist; + } + ret = rte_kvargs_process(kvlist, ETH_AF_XDP_PMD_ZC_ARG, &parse_integer_arg, pmd_zc); if (ret < 0) @@ -874,8 +919,8 @@ get_iface_info(const char *if_name, } static struct rte_eth_dev * -init_internals(struct rte_vdev_device *dev, const char *if_name, int queue_idx, - int pmd_zc) +init_internals(struct rte_vdev_device *dev, const char *if_name, + int start_queue_idx, int queue_cnt, int pmd_zc) { const char *name = rte_vdev_device_name(dev); const unsigned int numa_node = dev->device.numa_node; @@ -888,23 +933,54 @@ init_internals(struct rte_vdev_device *dev, const char *if_name, int queue_idx, if (internals == NULL) return NULL; - internals->queue_idx = queue_idx; + internals->start_queue_idx = start_queue_idx; + internals->queue_cnt = queue_cnt; internals->pmd_zc = pmd_zc; strlcpy(internals->if_name, if_name, IFNAMSIZ); - for (i = 0; i < ETH_AF_XDP_MAX_QUEUE_PAIRS; i++) { + if (xdp_get_channels_info(if_name, &internals->max_queue_cnt, + &internals->combined_queue_cnt)) { + AF_XDP_LOG(ERR, "Failed to get channel info of interface: %s\n", + if_name); + goto err_free_internals; + } + + if (queue_cnt > internals->combined_queue_cnt) { + AF_XDP_LOG(ERR, "Specified queue count %d is larger than combined queue count %d.\n", + queue_cnt, internals->combined_queue_cnt); + goto err_free_internals; + } + + internals->rx_queues = rte_zmalloc_socket(NULL, + sizeof(struct pkt_rx_queue) * queue_cnt, + 0, numa_node); + if (internals->rx_queues == NULL) { + AF_XDP_LOG(ERR, "Failed to allocate memory for rx queues.\n"); + goto err_free_internals; + } + + internals->tx_queues = rte_zmalloc_socket(NULL, + sizeof(struct pkt_tx_queue) * queue_cnt, + 0, numa_node); + if (internals->tx_queues == NULL) { + AF_XDP_LOG(ERR, "Failed to allocate memory for tx queues.\n"); + goto err_free_rx; + } + for (i = 0; i < queue_cnt; i++) { internals->tx_queues[i].pair = &internals->rx_queues[i]; internals->rx_queues[i].pair = &internals->tx_queues[i]; + internals->rx_queues[i].xsk_queue_idx = start_queue_idx + i; + internals->tx_queues[i].xsk_queue_idx = start_queue_idx + i; } ret = get_iface_info(if_name, &internals->eth_addr, &internals->if_index); if (ret) - goto err; + goto err_free_tx; eth_dev = rte_eth_vdev_allocate(dev, 0); if (eth_dev == NULL) - goto err; + goto err_free_tx; eth_dev->data->dev_private = internals; eth_dev->data->dev_link = pmd_link; @@ -920,7 +996,11 @@ init_internals(struct rte_vdev_device *dev, const char *if_name, int queue_idx, return eth_dev; -err: +err_free_tx: + rte_free(internals->tx_queues); +err_free_rx: + rte_free(internals->rx_queues); +err_free_internals: rte_free(internals); return NULL; } @@ -930,7 +1010,8 @@ rte_pmd_af_xdp_probe(struct rte_vdev_device *dev) { struct rte_kvargs *kvlist; char if_name[IFNAMSIZ] = {'\0'}; - int xsk_queue_idx = ETH_AF_XDP_DFLT_QUEUE_IDX; + int xsk_start_queue_idx = ETH_AF_XDP_DFLT_START_QUEUE_IDX; + int xsk_queue_cnt = ETH_AF_XDP_DFLT_QUEUE_COUNT; struct rte_eth_dev *eth_dev = NULL; const char *name; int pmd_zc = 0; @@ -960,7 +1041,8 @@ rte_pmd_af_xdp_probe(struct rte_vdev_device *dev) if (dev->device.numa_node == SOCKET_ID_ANY) dev->device.numa_node = rte_socket_id(); - if (parse_parameters(kvlist, if_name, &xsk_queue_idx, &pmd_zc) < 0) { + if (parse_parameters(kvlist, if_name, &xsk_start_queue_idx, + &xsk_queue_cnt, &pmd_zc) < 0) { AF_XDP_LOG(ERR, "Invalid kvargs value\n"); return -EINVAL; } @@ -970,7 +1052,8 @@ rte_pmd_af_xdp_probe(struct rte_vdev_device *dev) return -EINVAL; } - eth_dev = init_internals(dev, if_name, xsk_queue_idx, pmd_zc); + eth_dev = init_internals(dev, if_name, xsk_start_queue_idx, + xsk_queue_cnt, pmd_zc); if (eth_dev == NULL) { AF_XDP_LOG(ERR, "Failed to init internals\n"); return -1; @@ -1012,7 +1095,8 @@ static struct rte_vdev_driver pmd_af_xdp_drv = { RTE_PMD_REGISTER_VDEV(net_af_xdp, pmd_af_xdp_drv); RTE_PMD_REGISTER_PARAM_STRING(net_af_xdp, "iface=<string> " - "queue=<int> " + "start_queue=<int> " + "queue_count=<int> " "pmd_zero_copy=<0|1>"); RTE_INIT(af_xdp_init_log) -- 2.17.1 ^ permalink raw reply related [flat|nested] 16+ messages in thread
* Re: [dpdk-dev] [PATCH v2 2/3] net/af_xdp: add multi-queue support 2019-05-30 9:07 ` [dpdk-dev] [PATCH v2 2/3] net/af_xdp: add multi-queue support Xiaolong Ye @ 2019-05-30 15:32 ` Stephen Hemminger 2019-05-31 1:53 ` Ye Xiaolong 0 siblings, 1 reply; 16+ messages in thread From: Stephen Hemminger @ 2019-05-30 15:32 UTC (permalink / raw) To: Xiaolong Ye Cc: Qi Zhang, John McNamara, Marko Kovacevic, Karlsson Magnus, Topel Bjorn, dev On Thu, 30 May 2019 17:07:06 +0800 Xiaolong Ye <xiaolong.ye@intel.com> wrote: > This patch adds two parameters `start_queue` and `queue_count` to > specify the range of netdev queues used by AF_XDP pmd. > > Signed-off-by: Xiaolong Ye <xiaolong.ye@intel.com> Why does this have to be a config option, we already have max queues and number of queues in DPDK configuration. ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [dpdk-dev] [PATCH v2 2/3] net/af_xdp: add multi-queue support 2019-05-30 15:32 ` Stephen Hemminger @ 2019-05-31 1:53 ` Ye Xiaolong 0 siblings, 0 replies; 16+ messages in thread From: Ye Xiaolong @ 2019-05-31 1:53 UTC (permalink / raw) To: Stephen Hemminger Cc: Qi Zhang, John McNamara, Marko Kovacevic, Karlsson Magnus, Topel Bjorn, dev On 05/30, Stephen Hemminger wrote: >On Thu, 30 May 2019 17:07:06 +0800 >Xiaolong Ye <xiaolong.ye@intel.com> wrote: > >> This patch adds two parameters `start_queue` and `queue_count` to >> specify the range of netdev queues used by AF_XDP pmd. >> >> Signed-off-by: Xiaolong Ye <xiaolong.ye@intel.com> > >Why does this have to be a config option, we already have max queues >and number of queues in DPDK configuration. Here multi-queue refers to nic ethdev queue, AF_XDP now has 1:1 mapping between af_xdp socket and netdev queue, this patch is to support one vdev with multi af_xdp sockets (netdev queues). Thanks, Xiaolong ^ permalink raw reply [flat|nested] 16+ messages in thread
* [dpdk-dev] [PATCH v2 3/3] net/af_xdp: remove unused struct member 2019-05-30 9:07 ` [dpdk-dev] [PATCH v2 0/3] add more features for AF_XDP pmd Xiaolong Ye 2019-05-30 9:07 ` [dpdk-dev] [PATCH v2 1/3] net/af_xdp: enable zero copy by extbuf Xiaolong Ye 2019-05-30 9:07 ` [dpdk-dev] [PATCH v2 2/3] net/af_xdp: add multi-queue support Xiaolong Ye @ 2019-05-30 9:07 ` Xiaolong Ye 2019-06-10 16:54 ` [dpdk-dev] [PATCH v2 0/3] add more features for AF_XDP pmd Ferruh Yigit 3 siblings, 0 replies; 16+ messages in thread From: Xiaolong Ye @ 2019-05-30 9:07 UTC (permalink / raw) To: Xiaolong Ye, Qi Zhang; +Cc: Karlsson Magnus, Topel Bjorn, dev, stable Fixes: f1debd77efaf ("net/af_xdp: introduce AF_XDP PMD") Cc: stable@dpdk.org Signed-off-by: Xiaolong Ye <xiaolong.ye@intel.com> --- drivers/net/af_xdp/rte_eth_af_xdp.c | 1 - 1 file changed, 1 deletion(-) diff --git a/drivers/net/af_xdp/rte_eth_af_xdp.c b/drivers/net/af_xdp/rte_eth_af_xdp.c index f56aabcae..fc25d245b 100644 --- a/drivers/net/af_xdp/rte_eth_af_xdp.c +++ b/drivers/net/af_xdp/rte_eth_af_xdp.c @@ -118,7 +118,6 @@ struct pmd_internals { int pmd_zc; struct ether_addr eth_addr; - struct rte_mempool *mb_pool_share; struct pkt_rx_queue *rx_queues; struct pkt_tx_queue *tx_queues; -- 2.17.1 ^ permalink raw reply related [flat|nested] 16+ messages in thread
* Re: [dpdk-dev] [PATCH v2 0/3] add more features for AF_XDP pmd 2019-05-30 9:07 ` [dpdk-dev] [PATCH v2 0/3] add more features for AF_XDP pmd Xiaolong Ye ` (2 preceding siblings ...) 2019-05-30 9:07 ` [dpdk-dev] [PATCH v2 3/3] net/af_xdp: remove unused struct member Xiaolong Ye @ 2019-06-10 16:54 ` Ferruh Yigit 3 siblings, 0 replies; 16+ messages in thread From: Ferruh Yigit @ 2019-06-10 16:54 UTC (permalink / raw) To: Xiaolong Ye; +Cc: Qi Zhang, Karlsson Magnus, Topel Bjorn, dev On 5/30/2019 10:07 AM, Xiaolong Ye wrote: > This patch series mainly includes 3 new features for AF_XDP pmd. They > are separated independent features, the reason I take them in one > patchset is that they have code dependency. > > 1. zero copy > > This patch enables `zero copy` between af_xdp umem and mbuf by using > external mbuf mechanism. > > 2. multi-queue > > With mutli-queue support, one AF_XDP pmd instance can use multi netdev > queues. > > 3. busy-poll > > With busy-poll, all processing occurs on a single core, performance is > better from a per-core perspective. > > This patch has dependency on busy-poll support in kernel side and now it > is in > RFC stage [1]. > > [1] https://www.spinics.net/lists/netdev/msg568337.html > > V2 changes: > > 1. improve mutli-queue support by getting the ethtool channel, so the > driver is able to get a reason maximum queue number. > 2. add a tiny cleanup patch to get rid of unused struct member > 3. remove the busy-poll patch as its kernel counterpart changes, will > update the patch later > > Xiaolong Ye (3): > net/af_xdp: enable zero copy by extbuf > net/af_xdp: add multi-queue support > net/af_xdp: remove unused struct member Series applied to dpdk-next-net/master, thanks. ^ permalink raw reply [flat|nested] 16+ messages in thread
end of thread, other threads:[~2019-06-13 0:33 UTC | newest] Thread overview: 16+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2019-05-15 8:38 [dpdk-dev] [PATCH v1 0/3] add more features for AF_XDP pmd Xiaolong Ye 2019-05-15 8:38 ` [dpdk-dev] [PATCH v1 1/3] net/af_xdp: enable zero copy by extbuf Xiaolong Ye 2019-05-15 8:38 ` [dpdk-dev] [PATCH v1 2/3] net/af_xdp: add multi-queue support Xiaolong Ye 2019-05-15 8:38 ` [dpdk-dev] [PATCH v1 3/3] net/af_xdp: add busy poll support Xiaolong Ye 2019-05-30 9:07 ` [dpdk-dev] [PATCH v2 0/3] add more features for AF_XDP pmd Xiaolong Ye 2019-05-30 9:07 ` [dpdk-dev] [PATCH v2 1/3] net/af_xdp: enable zero copy by extbuf Xiaolong Ye 2019-05-30 15:31 ` Stephen Hemminger 2019-05-31 1:49 ` Ye Xiaolong 2019-06-11 16:16 ` William Tu 2019-06-12 10:03 ` Ye Xiaolong 2019-06-13 0:32 ` William Tu 2019-05-30 9:07 ` [dpdk-dev] [PATCH v2 2/3] net/af_xdp: add multi-queue support Xiaolong Ye 2019-05-30 15:32 ` Stephen Hemminger 2019-05-31 1:53 ` Ye Xiaolong 2019-05-30 9:07 ` [dpdk-dev] [PATCH v2 3/3] net/af_xdp: remove unused struct member Xiaolong Ye 2019-06-10 16:54 ` [dpdk-dev] [PATCH v2 0/3] add more features for AF_XDP pmd Ferruh Yigit
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).