Re: [RFC v3] flow_classify: add librte_flow_classify library

["Iremonger, Bernard" <bernard.iremonger@intel.com>]

Hi Ferruh,

> -----Original Message-----
> From: Yigit, Ferruh
> Sent: Thursday, May 25, 2017 4:47 PM
> To: dev@dpdk.org
> Cc: Yigit, Ferruh <ferruh.yigit@intel.com>; Mcnamara, John
> <john.mcnamara@intel.com>; Tahhan, Maryam
> <maryam.tahhan@intel.com>; Iremonger, Bernard
> <bernard.iremonger@intel.com>; Adrien Mazarguil
> <adrien.mazarguil@6wind.com>
> Subject: [RFC v3] flow_classify: add librte_flow_classify library
> 
> Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
> ---
> Cc: Bernard Iremonger <bernard.iremonger@intel.com>
> Cc: Adrien Mazarguil <adrien.mazarguil@6wind.com>
> 
> RFC v3:
> * add create() / destroy() APIs
> * query() gets rte_flow_classify object as param
> * query() gets one flow at a time
> 
> RFC v2:
> * prefer user called functions to callbacks
> * use rte_flow to define flows and actions
> ---
>  config/common_base                                 |   5 +
>  doc/api/doxy-api-index.md                          |   1 +
>  doc/api/doxy-api.conf                              |   1 +
>  doc/guides/rel_notes/release_17_08.rst             |   1 +
>  lib/Makefile                                       |   2 +
>  lib/librte_flow_classify/Makefile                  |  50 +++++++
>  lib/librte_flow_classify/rte_flow_classify.c       | 153
> +++++++++++++++++++++
>  lib/librte_flow_classify/rte_flow_classify.h       | 149
> ++++++++++++++++++++
>  .../rte_flow_classify_version.map                  |   9 ++
>  mk/rte.app.mk                                      |   1 +
>  10 files changed, 372 insertions(+)
>  create mode 100644 lib/librte_flow_classify/Makefile  create mode 100644
> lib/librte_flow_classify/rte_flow_classify.c
>  create mode 100644 lib/librte_flow_classify/rte_flow_classify.h
>  create mode 100644 lib/librte_flow_classify/rte_flow_classify_version.map
> 
> diff --git a/config/common_base b/config/common_base index
> 8907bea..3a7e73a 100644
> --- a/config/common_base
> +++ b/config/common_base
> @@ -651,6 +651,11 @@ CONFIG_RTE_LIBRTE_IP_FRAG_TBL_STAT=n
>  CONFIG_RTE_LIBRTE_METER=y
> 
>  #
> +# Compile librte_classify
> +#
> +CONFIG_RTE_LIBRTE_FLOW_CLASSIFY=y
> +
> +#
>  # Compile librte_sched
>  #
>  CONFIG_RTE_LIBRTE_SCHED=y
> diff --git a/doc/api/doxy-api-index.md b/doc/api/doxy-api-index.md index
> f5f1f19..d18c2b6 100644
> --- a/doc/api/doxy-api-index.md
> +++ b/doc/api/doxy-api-index.md
> @@ -98,6 +98,7 @@ There are many libraries, so their headers may be
> grouped by topics:
>    [LPM IPv4 route]     (@ref rte_lpm.h),
>    [LPM IPv6 route]     (@ref rte_lpm6.h),
>    [ACL]                (@ref rte_acl.h),
> +  [flow_classify]      (@ref rte_flow_classify.h),
>    [EFD]                (@ref rte_efd.h)
> 
>  - **QoS**:
> diff --git a/doc/api/doxy-api.conf b/doc/api/doxy-api.conf index
> ca9194f..94f3d0f 100644
> --- a/doc/api/doxy-api.conf
> +++ b/doc/api/doxy-api.conf
> @@ -46,6 +46,7 @@ INPUT                   = doc/api/doxy-api-index.md \
>                            lib/librte_efd \
>                            lib/librte_ether \
>                            lib/librte_eventdev \
> +                          lib/librte_flow_classify \
>                            lib/librte_hash \
>                            lib/librte_ip_frag \
>                            lib/librte_jobstats \ diff --git
> a/doc/guides/rel_notes/release_17_08.rst
> b/doc/guides/rel_notes/release_17_08.rst
> index 74aae10..6362bb2 100644
> --- a/doc/guides/rel_notes/release_17_08.rst
> +++ b/doc/guides/rel_notes/release_17_08.rst
> @@ -152,6 +152,7 @@ The libraries prepended with a plus sign were
> incremented in this version.
>       librte_distributor.so.1
>       librte_eal.so.4
>       librte_ethdev.so.6
> +   + librte_flow_classify.so.1
>       librte_hash.so.2
>       librte_ip_frag.so.1
>       librte_jobstats.so.1
> diff --git a/lib/Makefile b/lib/Makefile index 07e1fd0..e63cd61 100644
> --- a/lib/Makefile
> +++ b/lib/Makefile
> @@ -80,6 +80,8 @@ DIRS-$(CONFIG_RTE_LIBRTE_POWER) += librte_power
> DEPDIRS-librte_power := librte_eal
>  DIRS-$(CONFIG_RTE_LIBRTE_METER) += librte_meter  DEPDIRS-
> librte_meter := librte_eal
> +DIRS-$(CONFIG_RTE_LIBRTE_FLOW_CLASSIFY) += librte_flow_classify
> +DEPDIRS-librte_flow_classify := librte_eal librte_ether librte_net
>  DIRS-$(CONFIG_RTE_LIBRTE_SCHED) += librte_sched  DEPDIRS-librte_sched
> := librte_eal librte_mempool librte_mbuf librte_net  DEPDIRS-librte_sched
> += librte_timer diff --git a/lib/librte_flow_classify/Makefile
> b/lib/librte_flow_classify/Makefile
> new file mode 100644
> index 0000000..c57e9a3
> --- /dev/null
> +++ b/lib/librte_flow_classify/Makefile
> @@ -0,0 +1,50 @@
> +#   BSD LICENSE
> +#
> +#   Copyright(c) 2017 Intel Corporation. All rights reserved.
> +#   All rights reserved.
> +#
> +#   Redistribution and use in source and binary forms, with or without
> +#   modification, are permitted provided that the following conditions
> +#   are met:
> +#
> +#     * Redistributions of source code must retain the above copyright
> +#       notice, this list of conditions and the following disclaimer.
> +#     * Redistributions in binary form must reproduce the above copyright
> +#       notice, this list of conditions and the following disclaimer in
> +#       the documentation and/or other materials provided with the
> +#       distribution.
> +#     * Neither the name of Intel Corporation nor the names of its
> +#       contributors may be used to endorse or promote products derived
> +#       from this software without specific prior written permission.
> +#
> +#   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND
> CONTRIBUTORS
> +#   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT
> NOT
> +#   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
> FITNESS FOR
> +#   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
> COPYRIGHT
> +#   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
> INCIDENTAL,
> +#   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
> NOT
> +#   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS
> OF USE,
> +#   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED
> AND ON ANY
> +#   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR
> TORT
> +#   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
> THE USE
> +#   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH
> DAMAGE.
> +
> +include $(RTE_SDK)/mk/rte.vars.mk
> +
> +# library name
> +LIB = librte_flow_classify.a
> +
> +CFLAGS += -O3
> +CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR)
> +
> +EXPORT_MAP := rte_flow_classify_version.map
> +
> +LIBABIVER := 1
> +
> +# all source are stored in SRCS-y
> +SRCS-$(CONFIG_RTE_LIBRTE_FLOW_CLASSIFY) := rte_flow_classify.c
> +
> +# install this header file
> +SYMLINK-$(CONFIG_RTE_LIBRTE_FLOW_CLASSIFY)-include :=
> +rte_flow_classify.h
> +
> +include $(RTE_SDK)/mk/rte.lib.mk
> diff --git a/lib/librte_flow_classify/rte_flow_classify.c
> b/lib/librte_flow_classify/rte_flow_classify.c
> new file mode 100644
> index 0000000..7c99c88
> --- /dev/null
> +++ b/lib/librte_flow_classify/rte_flow_classify.c
> @@ -0,0 +1,153 @@
> +/*-
> + *   BSD LICENSE
> + *
> + *   Copyright(c) 2017 Intel Corporation. All rights reserved.
> + *   All rights reserved.
> + *
> + *   Redistribution and use in source and binary forms, with or without
> + *   modification, are permitted provided that the following conditions
> + *   are met:
> + *
> + *     * Redistributions of source code must retain the above copyright
> + *       notice, this list of conditions and the following disclaimer.
> + *     * Redistributions in binary form must reproduce the above copyright
> + *       notice, this list of conditions and the following disclaimer in
> + *       the documentation and/or other materials provided with the
> + *       distribution.
> + *     * Neither the name of Intel Corporation nor the names of its
> + *       contributors may be used to endorse or promote products derived
> + *       from this software without specific prior written permission.
> + *
> + *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND
> CONTRIBUTORS
> + *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT
> NOT
> + *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
> FITNESS FOR
> + *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
> COPYRIGHT
> + *   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
> INCIDENTAL,
> + *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
> NOT
> + *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS
> OF USE,
> + *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED
> AND ON ANY
> + *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR
> TORT
> + *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
> THE USE
> + *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH
> DAMAGE.
> + */
> +
> +#include <rte_flow_classify.h>
> +
> +LIST_HEAD(rte_flow_classify_list, rte_flow_classify) rte_flow_classify_head
> =
> +	LIST_HEAD_INITIALIZER(rte_flow_classify_head);
> +
> +struct rte_flow_classify {
> +	LIST_ENTRY(rte_flow_classify) next;
> +	struct rte_flow_action action;
> +	uint32_t id;
> +};
> +
> +static uint32_t unique_id = 1;
> +
> +static struct rte_flow_classify *
> +allocate(const struct rte_flow_action *action) {
> +
> +	struct rte_flow_classify *flow_classify = NULL;
> +
> +	flow_classify = malloc(sizeof(struct rte_flow_classify));
> +
> +	if (!flow_classify)
> +		return flow_classify;
> +
> +	flow_classify->action = *action;
> +	flow_classify->id = unique_id++;
> +
> +	return flow_classify;
> +}
> +
> +struct rte_flow_classify *
> +rte_flow_classify_create(const struct rte_flow_attr *attr,
> +		const struct rte_flow_item pattern[],
> +		const struct rte_flow_action *action) {
> +	struct rte_flow_classify *flow_classify;
> +
> +	if (!attr || !pattern || !action)
> +		return NULL;
> +
> +	flow_classify = allocate(action);
> +	if (!flow_classify)
> +		return NULL;
> +
> +	/* parse attr, pattern and action,
> +	 * create mask or hash values etc to match flow easier
> +	 * update flow_classify object to include these data */
> +
> +	LIST_INSERT_HEAD(&rte_flow_classify_head, flow_classify, next);
> +
> +	return flow_classify;
> +}
> +
> +int
> +rte_flow_classify_destroy(struct rte_flow_classify *flow_classify) {
> +	if (!flow_classify)
> +		return -EINVAL;
> +
> +	LIST_REMOVE(flow_classify, next);
> +
> +	free(flow_classify);
> +
> +	return 0;
> +}
> +
> +static int
> +flow_match(const struct rte_flow_classify *flow_classify,
> +		const struct rte_mbuf *m)
> +{
> +	(void)flow_classify;
> +	(void)m;
> +
> +	return 0;
> +}
> +
> +static int
> +action_apply(const struct rte_flow_classify *flow_classify,
> +		const struct rte_mbuf *m,
> +		struct rte_flow_classify_stats *stats) {
> +	switch (flow_classify->action.type) {
> +	default:
> +		return -ENOTSUP;
> +	}
> +
> +	stats->used_space++;
> +
> +	(void)m;
> +
> +	return 0;
> +}
> +
> +int
> +rte_flow_classify_query(const struct rte_flow_classify *flow_classify,
> +		struct rte_mbuf **pkts,
> +		const uint16_t nb_pkts,
> +		struct rte_flow_classify_stats *stats) {
> +	const struct rte_mbuf *m;
> +	int ret = 0;
> +	uint16_t i;
> +
> +	if (stats->available_space == 0)
> +		return -EINVAL;
> +
> +	stats->used_space = 0;
> +
> +	for (i = 0; i < nb_pkts; i++) {
> +		m = pkts[i];
> +
> +		if (!flow_match(flow_classify, m)) {
> +			ret = action_apply(flow_classify, m, stats);
> +			if (ret)
> +				break;
> +		}
> +	}
> +
> +	return ret;
> +}
> diff --git a/lib/librte_flow_classify/rte_flow_classify.h
> b/lib/librte_flow_classify/rte_flow_classify.h
> new file mode 100644
> index 0000000..5440775
> --- /dev/null
> +++ b/lib/librte_flow_classify/rte_flow_classify.h
> @@ -0,0 +1,149 @@
> +/*-
> + *   BSD LICENSE
> + *
> + *   Copyright(c) 2017 Intel Corporation. All rights reserved.
> + *   All rights reserved.
> + *
> + *   Redistribution and use in source and binary forms, with or without
> + *   modification, are permitted provided that the following conditions
> + *   are met:
> + *
> + *     * Redistributions of source code must retain the above copyright
> + *       notice, this list of conditions and the following disclaimer.
> + *     * Redistributions in binary form must reproduce the above copyright
> + *       notice, this list of conditions and the following disclaimer in
> + *       the documentation and/or other materials provided with the
> + *       distribution.
> + *     * Neither the name of Intel Corporation nor the names of its
> + *       contributors may be used to endorse or promote products derived
> + *       from this software without specific prior written permission.
> + *
> + *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND
> CONTRIBUTORS
> + *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT
> NOT
> + *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
> FITNESS FOR
> + *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
> COPYRIGHT
> + *   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
> INCIDENTAL,
> + *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
> NOT
> + *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS
> OF USE,
> + *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED
> AND ON ANY
> + *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR
> TORT
> + *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
> THE USE
> + *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH
> DAMAGE.
> + */
> +
> +#ifndef _RTE_FLOW_CLASSIFY_H_
> +#define _RTE_FLOW_CLASSIFY_H_
> +
> +/**
> + * @file
> + *
> + * RTE Flow Classify Library
> + *
> + * This library provides flow record information with some measured
> properties.
> + *
> + * Application should define the flow and measurement criteria (action) for
> it.
> + *
> + * Library doesn't maintain any flow records itself, instead flow
> +information is
> + * returned to upper layer only for given packets.
> + *
> + * It is application's responsibility to call rte_flow_classify_query()
> + * for group of packets, just after receive them or before transmit them.
> + * Application should provide the flow type interested in, measurement
> +to apply
> + * that flow in rte_flow_classify_create() API, and should provide
> + * rte_flow_classify object and storage to put results in
> + * rte_flow_classify_query() API.
> + *
> + *  Usage:
> + *  - application calls rte_flow_classify_create() to create a rte_flow_classify
> + *    object.
> + *  - application calls rte_flow_classify_query() in a polling manner,
> + *    preferably after rte_eth_rx_burst(). This will cause the library to
> + *    convert packet information to flow information with some
> measurements.
> + *  - rte_flow_classify object can be destroyed when they are no more
> needed via
> + *    rte_flow_classify_destroy()
> + */
> +
> +#include <rte_ethdev.h>
> +#include <rte_ether.h>
> +#include <rte_flow.h>
> +
> +#ifdef __cplusplus
> +extern "C" {
> +#endif
> +
> +struct rte_flow_classify;
> +
> +/**
> + * Flow stats
> + *
> + * For single action an array of stats can be returned by API.
> +Technically each
> + * packet can return a stat at max.
> + *
> + * Storage for stats is provided by application, library should know
> +available
> + * space, and should return the number of used space.
> + *
> + * stats type is based on what measurement (action) requested by
> application.
> + *
> + */
> +struct rte_flow_classify_stats {
> +	const unsigned int available_space;
> +	unsigned int used_space;
> +	void **stats;
> +};
> +
> +/**
> + * Create a flow classify rule.
> + *
> + * @param[in] attr
> + *   Flow rule attributes
> + * @param[in] pattern
> + *   Pattern specification (list terminated by the END pattern item).
> + * @param[in] action
> + *   Associated action
> + *
> + * @return
> + *   A valid handle in case of success, NULL otherwise.
> + */
> +struct rte_flow_classify *
> +rte_flow_classify_create(const struct rte_flow_attr *attr,
> +		const struct rte_flow_item pattern[],
> +		const struct rte_flow_action *action);
> +
> +/**
> + * Destroy a flow classify rule.
> + *
> + * @param flow_classify
> + *   Flow rule handle to destroy
> + *
> + * @return
> + *   0 on success, a negative errno value otherwise.
> + */
> +int
> +rte_flow_classify_destroy(struct rte_flow_classify *flow_classify);
> +
> +/**
> + * Get flow classification stats for given packets.
> + *
> + * @param flow_classify
> + *   Flow rule object
> + * @param pkts
> + *   Pointer to packets to process
> + * @param nb_pkts
> + *   Number of packets to process
> + * @param stats
> + *   To store stats define by action
> + *
> + * @return
> + *   0 on success, a negative errno value otherwise.
> + */
> +int
> +rte_flow_classify_query(const struct rte_flow_classify *flow_classify,
> +		struct rte_mbuf **pkts,
> +		const uint16_t nb_pkts,
> +		struct rte_flow_classify_stats *stats);
> +
> +#ifdef __cplusplus
> +}
> +#endif
> +
> +#endif /* _RTE_FLOW_CLASSIFY_H_ */
> diff --git a/lib/librte_flow_classify/rte_flow_classify_version.map
> b/lib/librte_flow_classify/rte_flow_classify_version.map
> new file mode 100644
> index 0000000..5aaf664
> --- /dev/null
> +++ b/lib/librte_flow_classify/rte_flow_classify_version.map
> @@ -0,0 +1,9 @@
> +DPDK_17.08 {
> +	global:
> +
> +	rte_flow_classify_create;
> +	rte_flow_classify_destroy;
> +	rte_flow_classify_query;
> +
> +	local: *;
> +};
> diff --git a/mk/rte.app.mk b/mk/rte.app.mk index bcaf1b3..adb6be4 100644
> --- a/mk/rte.app.mk
> +++ b/mk/rte.app.mk
> @@ -81,6 +81,7 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_POWER)          += -
> lrte_power
>  _LDLIBS-$(CONFIG_RTE_LIBRTE_TIMER)          += -lrte_timer
>  _LDLIBS-$(CONFIG_RTE_LIBRTE_EFD)            += -lrte_efd
>  _LDLIBS-$(CONFIG_RTE_LIBRTE_CFGFILE)        += -lrte_cfgfile
> +_LDLIBS-$(CONFIG_RTE_LIBRTE_FLOW_CLASSIFY) += -lrte_flow_classify
> 
>  _LDLIBS-y += --whole-archive
> 
> --
> 2.9.3

It is probably useful to add an rte_flow_classify_validate API .

Regards,

Bernard.